Usability Flaws Research Articles

Paper-and-Pencil vs. Electronic Patient Records: Analyzing Time Efficiency, Personnel Requirements, and Usability Impacts on Healthcare Administration.

Background: This study investigates the impact of transitioning from paper and pencil (P&P) methods to electronic patient records (EPR) on workflow and usability in surgical ward rounds. Methods: Surgical ward rounds were audited by two independent observers to evaluate the effects of transitioning from P&P to EPR. Key observations included the number of medical personnel and five critical workflow aspects before and after EPR implementation. Additionally, usability was assessed using the System Usability Scale (SUS) and the Post-Study System Usability Questionnaire (PSSUQ). Results: A total of 192 P&P and 160 EPR observations were analyzed. Physicians experienced increased administrative workload with EPR, while nurses adapted more easily. Ward teams typically consisted of two physicians and three or four nurses. Usability scores rated the system as "Not Acceptable" across all professional groups. Conclusions: The EPR system introduced usability challenges, particularly for physicians, despite potential benefits like improved data access. Usability flaws hindered system acceptance, highlighting the need for better workflow integration. Addressing these issues could improve efficiency and reduce administrative strain. As artificial intelligence becomes more integrated into clinical practice, healthcare professionals must critically assess AI-driven tools to ensure safe and effective patient care.

Electronic maternal and child health application usability, feasibility and acceptability among healthcare providers in Amhara region, Ethiopia.

An innovative electronic Maternal and Child Health (eMCH) application was developed to support operational and clinical decision-making in maternal and child health services. End-user-based evaluation of eHealth application is a critical step to ascertain how successfully users can learn and use it, and improve the technology. Therefore, this study aimed to evaluate the eMCH tool usability, feasibility, and acceptability among healthcare providers (HCPs) in the Amhara region, Ethiopia. A cross-sectional study was conducted among HCPs working in six public healthcare facilities. The usability evaluation was done on 24 HCPs across three professional categories using the ISO 9241-11 usability guideline. One hundred nine HCPs were participated in the feasibility and acceptability study. Data were collected using a standard usability tool, think-aloud protocol, a self-administered approach, and Open Broadcaster Software Studio version 26.1.1 video recorder. Descriptive statistics were used to describe the data. A Kruskal-Wallis test was used to measure the association between mean scores and categories of HCPs. The recorded videos were used for the log file analysis method. None of the HCP categories were able to complete all the tasks without errors. The average number of errors and restarts were 7.5 and 2.8, respectively. The average number of restarts was directly proportional to the average number of errors. The participants successfully completed more than 70% of the tasks without requiring any assistance or guidance. Forty-seven comments or errors were identified from the think-aloud analysis and 22 comments from the usability metrics analysis. Overall, statistically significant performance differences were observed among the three HCP groups across the majority of the usability evaluation metrics. Fifty-seven percent of HCPs scored higher than the mean on the feasibility study. Slightly higher than half, 56 (51.4%), of the HCPs scored higher than the mean score on the acceptability study. The usability evaluation identified vital comments and usability flaws that were essential for the eMCH tool to be upgraded. The tool was feasible and acceptable as reported by end-users. Therefore, the errors and usability flaws of the tool should be fixed before deployment to other healthcare settings.

Usability of Standards for Scaffolding in a Health Sciences Programme: A feasibility Study

BackgroundStandards contribute to comprehensive and programmatic implementation of educational strategies, such as scaffolding. Although the development of educational standards follows a rigorous consensus approach, they are socially constructed and could result in varied interpretations by users. Reports of varied implementation of standards in health professions education underscore the need to test the developed standards for scaffolding in health sciences programmes. Usability entails determining whether a product like standards works as intended under the expected conditions and contexts. This study aimed to describe the usability of standards for scaffolding in a health sciences programme through a pilot study.MethodsA multi-method design employing user and expert-based usability evaluation techniques sought to describe the usability of the standards for scaffolding in a three-year pre-registration nursing programme. The user sample of nurse educators drawn from the programme, conducted a self-assessment on scaffolding practices in the programme using a developed standards checklist. For the expert sample, three-panel members with an understanding of the discipline and programme context were purposively sampled. These panelists studied the users’ self-assessment reports before completing an author-generated heuristics checklist to support or refute any of the standards. Descriptive statistics, comparative and content analysis were applied to analyse data from users’ interviews and expert’s completed heuristics checklist, determining the standards’ usability, and identifying the usability flaws or strengths.ResultsThe users had three or more years of teaching experience in the competency-based curriculum for nursing. The experts shared an average of 16 years of experience in teaching in higher education, and seven years of experience in quality assurance and programme accreditation. The four standards had a usability score of above average (68%). Seven usability strengths and four usability flaws were identified. Usability flaws related to misinterpretation of some criteria statements and terminologies, multiple meanings, and users’ challenges in generating evidence for some criteria.ConclusionsThe pilot study revealed the context-based ‘truth’ regarding the fidelity of a health sciences programme evaluation on scaffolding, as well as identifying the ideal contextual conditions in which the standards for scaffolding health sciences programmes would work best. The identified usability flaws highlighted the need for further revisions of the standards. Future research on the feasibility of the standards in other health sciences programmes and contexts is recommended.

Exploring User Experience and Usability in a Metaverse Learning Environment for Students: A Usability Study of the Artificial Intelligence, Innovation, and Society (AIIS)

The metaverse, a rapidly evolving research area, is a virtual, interconnected universe that seamlessly integrates digital and physical realms, facilitating immersive experiences, social interaction, and economic activities across various fields, including computer science, business, and education. Despite its potential to transform current pedagogies and learning experiences through engaging, interactive, and fascinating educational techniques, research is limited in exploring its application, usability, and user experience, particularly for Thai undergraduate students in learning computer engineering courses. In response, we designed and developed an innovative metaverse-based learning system called the AIIS collaborative learning interface, tailored to European undergraduate students. While the original intent of AIIS was to cater to medical students, in this study, our objective was to assess its usability and applicability for computer science and engineering students, specifically focusing on the Asian demographic. After an initial pilot with European students in their local context, the research was extended to Thailand, where 21 Thai undergraduate students evaluated the platform’s usability. The observations from this study indicate that AIIS provides a dynamic, user-friendly learning environment. Nevertheless, the evaluation process unveiled some usability flaws that offer informative directions for future enhancements. Drawing from our observations, we formulated usability guidelines to guide designers and developers toward crafting more efficient metaverse systems, particularly those intended for educational use. Our results also underline the considerable promise that metaverse technologies hold for advancing higher education globally.

Assessing Heuristic Evaluation in Immersive Virtual Reality—A Case Study on Future Guidance Systems

A variety of evaluation methods for user interfaces (UI) exist such as usability testing, cognitive walkthrough, and heuristic evaluation. However, UIs such as guidance systems at transit hubs must be evaluated in their intended application field to allow the effective and valid identification of usability flaws. However, what if evaluations are not feasible in real environments, or laboratorial conditions cannot be ensured? Based on adapted heuristics, in the present study, the method of heuristic evaluation is combined with immersive Virtual Reality (VR) for the identification of usability flaws of dynamic guidance systems (DGS) at transit hubs. The study involved usability evaluations of nine DGS concepts using the newly proposed method. The results show that compared to computer-based heuristic evaluations, the use of immersive VR led to the identification of an increased amount of “severe” usability flaws as well as overall usability flaws. Within a qualitative assessment, immersive VR is validated as a suitable tool for conducting heuristic evaluations involving significant advantages such as the creation of realistic experiences in laboratorial conditions. Future work seeks to further prove the suitability of using immersive VR for heuristic evaluations and compare the proposed method to other evaluative methods.

Design and Development of a User-Centered Mobile Application for Intermodal Public Transit in Bangkok: A Design Thinking Approach

With the constant increase in public transit coverage in Bangkok Metropolitan Area in Thailand, many people are still hesitant to switch from using private to public transit, with one potential cause being the unavailability and difficulty in accessing accurate and timely information about their itineraries, while they are commuting. To assess and tackle such issues, the researchers adopted the user-centric Design Thinking methodology to empathize with target users' pain points in this study. They proposed a solution in a user-centric manner by assessing the usability flaws of existing mobile navigation applications, such as Google Maps and ViaBus. By developing a holistic mobile application called 'Disgovery' that covers all modes of public transit in the Bangkok Metropolitan Area and provides relevant information about their trips, the researchers aim to help commuters in Bangkok easily access it in a timely fashion. Through the user-friendly interface, commuters can eliminate the difficulty of finding routes and prices suitable to their needs. By making public transit more accessible with the help of ubiquitous mobile computing, commuters are also encouraged to switch from using private vehicles to public transit, which also can reduce accidents and carbon emissions. The findings from the usability testing in this study suggest that 'Disgovery' is an effective and user-friendly application for daily commuters in Bangkok that can help them achieve their goals without difficulties. The findings also indicate the importance of user interface and user experience guidelines in designing such applications.

Tycho: Facilitation Support for Groupware User Tests

Running user tests for groupware requires tracking the progress of multiple users performing co-dependent tasks, while facilitating and observing their coordinated actions. This requirement negatively impacts the main objective of user testing, which is detecting usability flaws. User testing of groupware becomes more challenging when run remotely. Even if there are tools for remote user testing that can get the job done for a single user, they are not prepared for collaborative scenarios. In this paper we argue that tool support for the facilitation of user tests of groupware is missing. Consequently, we propose a method for user tests with tool assistance that makes it possible to automate task synchronization, especially for different workflows that must be run concurrently. We evaluated our proposal by comparing it to a manually facilitated approach during on-site coordinated user tests. The results indicate that, while the task of designing and running user tests with tool support takes more time, it allows the testers to better focus on detecting usability problems.

Applying usability recommendations when developing mobile instant messaging applications

Mobile Instant Messaging applications noted a growing number of active users recently. Although having millions of users, the previous research results have shown usability flaws in this type of mobile applications. Difficulties when performing tasks, poor design of user interfaces, or lack of information about privacy and security methods were some of the identified flaws. Consequently, a collection of usability recommendations was proposed. The goal of these recommendations is to improve the experience in the use of these mobile applications and, thus, enhancing the usability of this kind of apps. These recommendations were taken into account during the development process of a new instant messaging application. Thus, in this study, the authors present the resulting application, its characteristics, how usability recommendations were covered through design and development phases, and a preliminary usability evaluation performed by conducting two research methods: KLM and Mobile Heuristic Evaluation. Both methods ranked the prototype at top positions (i.e. good usability results) when compared with the results obtained in previous studies about other existing instant messaging applications.

Designing Shift Handoff Software: Clinical Learners and Design Students Collaborate Using the "Design Thinking" Process.

Handoffs in patient care responsibilities between practitioners are common in the hospital setting. Because inadequate communication can lead to patient harm, professional organizations have published recommendations and practical guides to support standardized workflow. However, currently available electronic medical record (EMR) tools rarely provide the requisite functionality to support work and often suffer from major usability flaws. Our internal medicine residency program sponsored a quality improvement initiative to improve the design of handoff tools. To support this initiative, our medical informatics program collaborated with a school of architecture and design to identify requirements and ideate interface prototypes. In this article, we describe how we used Design Thinking principles and methods to inform our product design lifecycle, create novel designs, and teach inter-professional students health systems science concepts.

EAUHHCAMA: Extending and Adapting Usability Heuristics for Healthcare Related Context-Aware Mobile Applications

Fitness related context-aware mobile applications (FCAMA) being the fragment of healthcare informatics, use emerging technologies and provide continuous and real-time information to the end-users. This information supports the users in understanding the physical condition and the way to improve health using cues. That's why; healthcare informatics is the fast-growing domain of interest for researchers. FCAMA has collected information from inbuilt smartphone tools including accelerometer, GPS, microphone, camera, and others. FCAMA transformed the extracted data into a usable form and formulated individual plans to track and improve fitness. But, it is ambiguous whether the user interfaces of FCAMA is really effective in improving health and fitness. Although, several studies attempt to establish an effective usability evaluation mechanism, it still needs further work for comprehensively identifying usability flaws. Evaluators of FCAMA use inspection based heuristics set to identify user-related issues. In order to identify the usability issues comprehensively, it is important that heuristics are pertinent. In this paper, we propose an enhanced and adapted set of fourteen usability heuristics for FCAMA. Eleven heuristics were selected and adapted after cross-linking the usability flaws with the existing sets of heuristics. Usability flaws were captured in a user study with 850 students from the human-computer interaction (HCI) course. While we added two more against uncaptured usability flaws, to be able to adapt the heuristics set to the context awareness aspect. The FCAMA was validated using two evaluation studies conducted by students in (HCI) course, and second by usability experts. Results have justified the proposed heuristics both in terms of effectiveness and relevance.

Usability evaluation of component based software system using software metrics

Component Based Software Engineering (CBSE) provides a way to create a new Component Based Software System (CBSS) by utilizing the existing components. The primary reason for that is to minimize the software development time, cost and effort. CBSS also increases the component reusability. Due to these advantages, software industries are working on CBSS and continuously trying to provide quality product. Usability is one of the major quality factors for CBSS. It should be measured before delivering the software product to the customer, so that if there are any usability flaws, it can be removed by software development team. In this paper, work has been done to evaluate the usability of CBSS based on major usability sub-factors (learnability, operability, understandability and configurability). For this purpose, firstly software metrics are identified for each usability sub-factor and the value of each sub-factor is evaluated for a component based software project. Secondly, overall usability of the software project is evaluated by using the calculated value of each usability sub-factor. Usability for the same project was also evaluated using Fuzzy approach in MATLAB to validate the experimental work of this research paper. It was identified that the value of usability obtained from software metrics and fuzzy model was very similar. This research work will be useful for the software developer to evaluate the usability of any CBSS and will also help them to compare different version of any CBSS in term of their usability.

A framework for heuristic evaluation of mobile learning applications

Heuristic evaluation is a fast and cheap method that can be used to identify usability flaws in mobile learning applications. In this paper, we proposed a framework for heuristic evaluation of mobile learning applications to ensure that best practices are followed, hence saving time and effort in conducting a heuristic evaluation. Ten case studies on heuristic evaluation of mobile learning applications were selected, and an analysis was conducted to identify the best practices that were later incorporated into a framework. The framework was utilized to conduct heuristic evaluation to show the feasibility of our approach. The results indicate that it provides clear guidance, and it’s fast and easy to conduct heuristic evaluation.

Analysis and measurement of internal usability metrics through code annotations

Nowadays, usability can be meant as an important quality characteristic to be considered throughout the software development process. A great variety of usability techniques have been proposed so far, mostly intended to be applied during analysis, design, and final testing phases in software projects. However, little or no attention has been paid to the analysis and measurement of usability in the implementation phase. Most of the time, usability testing is traditionally executed in advanced stages. However, the detection of usability flaws during the implementation is of utmost importance to foresee and prevent problems in the utilization of the software and avoid significant cost increases. In this paper, we propose a feasible solution to analyze and measure usability metrics during the implementation phase. Specifically, we have developed a framework featuring code annotations that provides a systematic evaluation of the usability throughout the source code. These annotations are interpreted by an annotation processor to obtain valuable information and automatically calculate usability metrics at compile time. In addition, an evaluation with 32 participants has been carried out to demonstrate the effectiveness and efficiency of our approach in comparison with the manual process of analyzing and measuring internal usability metrics. Perceived satisfaction was also evaluated, demonstrating that our approach can be considered as a valuable tool for dealing with usability metrics during the implementation phase.

Building Usability Knowledge for Health Information Technology: A Usability-Oriented Analysis of Incident Reports.

The contribution of usability flaws to patient safety issues is acknowledged but not well-investigated. Free-text descriptions of incident reports may provide useful data to identify the connection between health information technology (HIT) usability flaws and patient safety. This article examines the feasibility of using incident reports about HIT to learn about the usability flaws that affect patient safety. We posed three questions: (1) To what extent can we gain knowledge about usability issues from incident reports? (2) What types of usability flaws, related usage problems, and negative outcomes are reported in incidents reports? (3) What are the reported usability issues that give rise to patient safety issues? A sample of 359 reports from the U.S. Food and Drug Administration Manufacturer and User Facility Device Experience database was examined. Descriptions of usability flaws, usage problems, and negative outcomes were extracted and categorized. A supplementary analysis was performed on the incidents which contained the full chain going from a usability flaw up to a patient safety issue to identify the usability issues that gave rise to patient safety incidents. A total of 249 reports were included. We found that incident reports can provide knowledge about usability flaws, usage problems, and negative outcomes. Thirty-six incidents report how usability flaws affected patient safety (ranging from incidents without consequence, to death) involving electronic patient scales, imaging systems, and HIT for medication management. The most significant class of involved usability flaws is related to the reliability, the understandability, and the availability of the clinical information. Incidents reports involving HIT are an exploitable source of information to learn about usability flaws and their effects on patient safety. Results can be used to convince all stakeholders involved in the HIT system lifecycle that usability should be considered seriously to prevent patient safety incidents.

EUHSA: Extending Usability Heuristics for Smartphone Application

Emerging development trends in smartphone computing assist the user to enhance skills, health improvement, earning opportunities, sharing ideas, providing entertainment, and so on. Appropriate transfer of these assistances demands a vibrant appearance to a smartphone user. End users emphasize easy-to-use, ergonomics, efficiency, and visual aesthetic, among others. Measuring usability issues allows for improving these features. An inspection-based heuristics set identifies the usability issues for improving the quality of smartphone interfaces. Using suitable heuristics is highly relevant. In this paper, we present an extended and relevant set of 14 Usability Heuristics for Smartphone (EUHSA) application, where 13 heuristics were selected after cross-linking the identified usability flaws with the previously proposed heuristics through a user study with 800 students from a human-computer interaction course. An additional heuristic is proposed against uncaptured usability flaws. Furthermore, the EUHSA was validated using an expert evaluation study. Findings were compared with the previously proposed Joyce and SMASH sets of heuristics. The results explicitly justify the EUHSA and prove its effectiveness in evaluating the usability issues.

Evaluating Usability of Serious Games for Therapeutic Purpose: A Study of ASAH-i

Usability has been determined to be connected to achieving predetermined practical goals by identifying usability problems during the development cycle of a product or system. In order for the next and future versions of the systems to be enhanced and improved, usability flaws and errors must first be identified. On the same stroke, identifying flaws encountered by the end users will indirectly steer other developers from committing the same mistakes. This paper reveals findings from a usability assessment conducted on the serious game, which was developed for therapeutic purpose known as ASAH-i, which was custom-made to develop and encourage the perception and cognitive skills for children with speech and language delay (CSLD). Two groups of users were formed, namely, test group and control group. Twelve participants were selected which include six CSLD as participants for test group and another six normal children for the control group. In a controlled environment, observation and think-aloud methods were applied to analyse ASAH-i. The data gathered were then evaluated accordingly to the task success, time of the user assignment accomplishment, error rates, and satisfaction in regards to the utilisation of ASAH-i. The result proved that there is no profound disputes between the two corresponding groups, where almost all participants for both groups can play ASAH-i prototype effectively since no major usability issues were found. With regards to competency or efficiency, the result reveals that the control group can play ASAH-i prototype more competently and efficiently than the test group. However, it is interesting to note that both group members also succeeded to play ASAH-i with very little mistakes. This factor yields a strong base that provides towards the group members’ contentment and enjoyment, in which most group members in both teams were very much happy and satisfied with ASAH-i. With this fact in mind, we propose an endorsement or recommendation study which includes details and emphasises on future development to enhance the usability of ASAH-i.

Evidence-based usability design principles for medication alerting systems

BackgroundUsability flaws in medication alerting systems may have a negative impact on clinical use and patient safety. In order to prevent the release of alerting systems that contain such flaws, it is necessary to provide designers and evaluators with evidence-based usability design principles. The objective of the present study was to develop a comprehensive, structured list of evidence-based usability design principles for medication alerting systems.MethodsNine sets of design principles for medication alerting systems were analyzed, summarized, and structured. We then matched the summarized principles with a list of usability flaws in order to determine the level of underlying evidence.ResultsFifty-eight principles were summarized from the literature and two additional principles were defined, so that each flaw was matched with a principle. We organized the 60 summarized usability design principles into 6 meta-principles, 38 principles, and 16 sub-principles. Only 15 principles were not matched with a usability flaw. The 6 meta-principles respectively covered the improvement of the signal-to-noise ratio, the support for collaborative working, the fit with a clinician’s workflow, the data display, the transparency of the alerting system, and the actionable tools to be provided within an alert.ConclusionsIt is possible to develop an evidence-based, structured, comprehensive list of usability design principles that are specific to medication alerting systems and are illustrated by the corresponding usability flaws. This list represents an improvement over the current literature. Each principle is now associated with the best available evidence of its violation. This knowledge may help to improve the usability of medication alerting systems and, ultimately, decrease the harmful consequences of the systems’ usability flaws.

Reusable Software Usability Specifications for mHealth Applications.

One of the key factors for the adoption of mobile technologies, and in particular of mobile health applications, is usability. A usable application will be easier to use and understand by users, and will improve user's interaction with it. This paper proposes a software requirements catalog for usable mobile health applications, which can be used for the development of new applications, or the evaluation of existing ones. The catalog is based on the main identified sources in literature on usability and mobile health applications. Our catalog was organized according to the ISO/IEC/IEEE 29148:2011 standard and follows the SIREN methodology to create reusable catalogs. The applicability of the catalog was verified by the creation of an audit method, which was used to perform the evaluation of a real app, S Health, application created by Samsung Electronics Co. The usability requirements catalog, along with the audit method, identified several usability flaws on the evaluated app, which scored 83%. Some flaws were detected in the app related to the navigation pattern. Some more issues related to the startup experience, empty screens or writing style were also found. The way a user navigates through an application improves or deteriorates user's experience with the application. We proposed a reusable usability catalog and an audit method. This proposal was used to evaluate a mobile health application. An audit report was created with the usability issues identified on the evaluated application.

Combining usability evaluations to highlight the chain that leads from usability flaws to usage problems and then negative outcomes

Poor usability of health technology is thought to diminish work system performance, increase error rates and, potentially, harm patients. The present study (i) used a combination of usability evaluation methods to highlight the chain that leads from usability flaws to usage problems experienced by users and, ultimately, to negative patient outcomes, and (ii) validated this approach by studying two different discharge summary production systems. To comply with quality guidelines, the process of drafting and sending discharge summaries is increasingly being automated. However, the usability of these systems may modify their impact (or the absence thereof) in terms of production times and quality, and must therefore be evaluated. Here, we applied three successive techniques for usability evaluation (heuristic evaluation, user testing and field observation) to two discharge summary production systems (underpinned by different technologies). The systems’ main usability flaws led respectively to an increase in the time need to produce a discharge summary and the risk of patient misidentification. Our results are discussed with regard to the possibility of linking the usability flaws, usage problems and the negative outcomes by successively applying three methods for evaluating usability (heuristic evaluation, user testing and in situ observations) throughout the system development life cycle.

Biomedical Informatics Investigator

The BMI Investigator is a computer human interface built in .Net which allows simultaneous query of structured data such as demographics, administrative codes, medications (coded in RxNorm), laboratory test results (coded in LOINC) and formerly unstructured data in clinical notes (coded in SNOMED CT). The ontology terms identified using SNOMED are all coded as either positive, negative or uncertain assertions. They are then where applicable built into compositional expressions and stored in both a graph database and a triple store. The SNOMED CT codes are stored in a NOSQL database, Berkley DB, and the structured data is stored in SQL using the OMOP/OHDSI format. The BMI investigator also lets you develop models for cohort selection (data driven recruitment to clinical trials) and automated retrospective research using genomic criteria and we are adding image feature data currently to the system. We performed a usability experiment and the users identified some usability flaws which were used to improve the software. Overall, the BMI Investigator was felt to be usable by subject matter experts. Next steps for the software are to integrate genomic criteria and image features into the query engine.