Framework For Usability Testing Research Articles

Enabling reliable usability assessment and comparative analysis of medical software: a comprehensive framework for multimodal biomedical imaging platforms

PurposeA literature review reveals that, at the moment, all usability tests for Software as a Medical Device (SaMD) are designed in compliance with international standards but it also reveals a lack of formalization in the implementation and administration of such usability tests, which prevents the comparison of results from different tests for the same class of SaMD. This study aims to provide a reproducible usability testing framework for SaMD to establish a standardized protocol which can ensure repeatability and comparisons of similar SaMD for the visualization of medical images and data.MethodsThe devised protocol aligns with international standards and literature recommendations for usability and human factors engineering. It encompasses participant selection, testing environments, equipment setup for various testing methods (HDMI vs. wireless), and hardware interfaces (keyboard/mouse vs. touchscreen), as well as the roles of the required testers. The protocol consists of two distinct sections: exploratory tasks and specific scenarios, to assess software functions and real-life tasks, respectively. Effectiveness and efficiency are evaluated using video analysis and a custom Stopwatch software, while user satisfaction is measured through post-test questionnaires.ResultsThe usability testing protocol was applied to a Multimodal Biomedical Imaging Platform All-in-One software developed by Imaginalis S.r.l. (Sesto Fiorentino, Italy) for validation. The results of the usability testing protocol applied to the case-study software demonstrate good values of software’s effectiveness and efficiency, along with user satisfaction supporting the prior heuristic evaluation. The outcomes confirm the robustness, applicability, and reproducibility of the usability testing protocol, aligning with best practices.ConclusionsThe proposed usability testing framework enables reliable usability assessment and comparative analysis of medical software. Furthermore, the obtained results can serve as a reference for assessing other biomedical imaging platforms under development or ready for release.

Software Usability Testing Using EEG-Based Emotion Detection and Deep Learning.

It is becoming increasingly attractive to detect human emotions using electroencephalography (EEG) brain signals. EEG is a reliable and cost-effective technology used to measure brain activities. This paper proposes an original framework for usability testing based on emotion detection using EEG signals, which can significantly affect software production and user satisfaction. This approach can provide an in-depth understanding of user satisfaction accurately and precisely, making it a valuable tool in software development. The proposed framework includes a recurrent neural network algorithm as a classifier, a feature extraction algorithm based on event-related desynchronization and event-related synchronization analysis, and a new method for selecting EEG sources adaptively for emotion recognition. The framework results are promising, achieving 92.13%, 92.67%, and 92.24% for the valence-arousal-dominance dimensions, respectively.

A Practical Usability Study Framework Using the SUS and the Affinity Diagram: A Case Study on the Online Roadshow Website

Online Roadshow has many benefits in promoting a concept or a product to the public. It is a new model with the same purpose as a physical roadshow but with higher scalability and flexibility in terms of time, location, target audience, resource utilization, and data collection capability. However, the prototype implementation of the new model has not been evaluated for its usability. As ISO standards defined, usability has three key elements: effectiveness, efficiency, and satisfaction. The effectiveness and efficiency are highly dependent on the different systems, while satisfaction is measurable by System Usability Scale (SUS). SUS is a quick and easy technique where the usability of a system can be assessed in a short time. However, the result of SUS is only a grading scale, with no information on the problematic areas. Hence, this paper introduces a practical framework that combines the SUS with the Affinity Diagram. With the intention of maintaining the simplicity and elegance of the SUS, an additional open-ended question is asked to assess the usability problem of the website. The proposed practical SUS plus one open-ended-question usability-testing framework was applied to the Online Roadshow website. Since the average SUS score for the Online Roadshow website from the test was relatively low, the Affinity Diagram was used to analyze the open-ended comments from the user. As a result, the practical usability framework identified the usability problems on the Online Roadshow website to assist the developers in improving the usability.

User requirement and usability testing framework for information system development: case study of financial institution

Due to the fact that financial institution has its limited competency in developing the information system, outsourcing is often used to support the information system development which same as the case study of the issuing documents system in financial institute. The financial institution has received customer’s complaints regarding the mistake of issuing documents caused by the system functionality and human error in operations. Thus, the purpose of this study is to design framework of user requirement and usability testing for information system development to prevent the error in system functionality and set some quality checking process of documents before delivery to customer to prevent mistakes from human error in work operation. System Usability Scale showed that after proposed the framework of user requirement and usability testing to employees who related with the system development, their level of satisfaction toward the framework and non-framework of user requirement and usability testing raise up to 30.93% and 20%, respectively. After six months of implemented quality checking process of documents revealed zero complaints from customers regarding to the mistake of issuing documents.

Design Of E-Document System With Digital Signature Using User Centered Design Method

UCD (User Centered Design) is a design philosophy that places the user at the center of a system development process. The UCD approach has been supported by various techniques, methods, tools, procedures and processes that help design interactive systems that are more user-centered. This study aims to apply UCD in the design of e-document systems with digital signatures to determine user requirements for system functionality and interfaces. To find out the extent of the application of the UCD, a measurement of the level of acceptance and support of prospective users is used by making a data collection instrument in the form of a questionnaire according to the usability testing framework by using a Likert scale to calculate the variables. The results of testing of potential users showed that 85% stated that the functionality and interface design of the e-document system with digital signatures and can be developed in the next stage.

Implementation of User Centered Design Method in Designing Android-based Journal Reminder Application

The main objective of this study was to implement UCD in designing an Android-based journal reminder application to determine the user’s needs for functionality and application interface. This study uses Data Collection Method and Application Design using UCD. To find out the result of the implementation UCD, measurements are used at the level of acceptance and support of prospective users by making a data collection instrument in the form of a questionnaire according to the usability testing framework using the Likert scale to calculate the variables. The results of testing of prospective users indicate that 84% of prospective users state that the functionality and interface design of the android-based journal reminder application can be accepted and can be developed in the next stage. This is concluded from the results of functional and interface design evaluations through questionnaires using a Linkert scale which shows that the design built is at a GOOD interval, which means that the functional design and interface of the journal reminder application can be accepted and can proceed to the next stage. However, in order to improve the accuracy of the development of the journal reminder applications, it is necessary to make a propototype based on the results of UCD modeling and design.

Extended Framework for Usability Testing in e-Navigation Systems

The paper presents the framework of usability testing of Electronic chart display and information system (ECDIS) equipment based on the International Maritime Organization (IMO)’s Guidelines on Software Quality Assurance and Human-centred Design for e-Navigation. By incorporating the eye tracking techniques into the procedure it was possible to measure the visual attention distribution and the cognitive work-load. The presented method could be used to evaluate usability of every e-navigation system, which is necessary to ensure that the seafarers are able to successfully perform primary operations of systems upgraded with e-navigation functions, regardless of the type and specifications of the system and users’ knowledge and experience with the system. The initial results are presented and discussed as the study is still ongoing.