Cognitive Walkthrough Research Articles

UI/UX Design of Ineffable Psychological Counseling Mobile Application Using Design Thinking Method

Application design is one of the important things in application development because the application that has a bad design will cause discomfort and confusion for users, especially in health applications, which have their own difficulties where application design must focus on what users need to be easy to understand, especially mental health application because there are still few application for mental health or self-care even though since the pandemic, mental health problems have increased drastically, this makes many people seek help in mental help. Design thinking is a method used to solve a problem where the solution comes from that user’s experiences or needs. The Design Thinking method consists of five stages: Empathize, Define, Ideate, Prototype, and Test. In the test step, testing was carried out using the Cognitive Walkthrough method with the help of the Maze tool with a learnability value of 97%, an error rate of 0.04, time-based efficiency of 0.05 task/second, and the MIUS value of each task were obtained a fairly high value indicating that design prototype is easy to use, easy to understand and efficient. While the MAUS score got a score of 94, which was included in the high level, indicating that the interface design was feasible to be implemented.

The poor usability of OpenLDAP Access Control Lists

The usability of Access Control Lists (ACLs) of a widely used enterprise software for directory information services called OpenLDAP is addressed. A directory service is used to store a variety of data such as employee information and passwords, and can be seen as a critical infrastructure component of an enterprise. Security and in particular, access control of such data is of paramount importance, and OpenLDAP provides ACLs for this purpose that an administrator can configure. The usability, that is, the ease with which a human administrator can express a policy in an ACL, is then an important issue because misconfigurations are known to be a major cause of security vulnerabilities. Motivated by public pronouncements regarding the poor usability of OpenLDAP ACLs, a systematic study towards evaluating their usability is carried out. The authors begin with a cognitive walkthrough, which identifies the broad issues, which then informs the design of an ethics-approved study of 50 human participants. This study reveals that indeed, even with a limited syntax, adequate training and a focus only on devising a policy from scratch, OpenLDAP ACLs suffer from poor usability. The data gathered from this study is analysed further, and more detailed observations are made such as those regarding the difference in difficulty for different kinds of policy goals, and the nature of errors human participants make with OpenLDAP ACLs. As such, this work makes an important contribution to enterprise security and provides important insights for a (re)design of ACLs, in particular for OpenLDAP.

Comparison of usability evaluation methods for a health information system: heuristic evaluation versus cognitive walkthrough method

BackgroundThere are differences of opinion regarding the selection of the most practical usability evaluation method among different methods. The present study aimed to compare two expert-based evaluation methods in order to assess a nursing module as the most widely used module of a Hospital Information System (HIS).MethodsFive independent evaluators used the Heuristic Evaluation (HE) and Cognitive Walkthrough (CW) methods to evaluate the nursing module of Shafa HIS. In this regard, the number and severity of the recognized problems according to the usability attributes were compared using two evaluation methods.ResultsThe HE and CW evaluation methods resulted in the identification of 104 and 24 unique problems, respectively, of which 33.3% of recognized problems in the CW evaluation method overlapped with the HE method. The average severity of the recognized problems was considered to be minor (2.34) in the HE method and major (2.77) in the CW evaluation method. There was a significant difference in terms of the total number and average severity of the recognized problems by these methods (P < 0.001). Based on the usability attribute, the HE method identified a larger number of problems concerning all usability attributes, and a significant difference was observed in terms of the number of recognized problems in both methods for all attributes except ‘memorability’. Also, there was a significant difference between the two methods based on the average severity of recognized problems only in terms of ‘learnability’.ConclusionThe HE method identified more problems with lower average severity while the CW was able to recognize fewer problems with higher average severity. Regarding the evaluation goal, the HE method was able to be used to improve the effectiveness and satisfaction of the HIS. Furthermore, the CW evaluation method is recommended to identify usability problems with the highest average severity, especially in terms of ‘learnability’.

Digitally Optimizing the Information Flows Necessary to Manage Professional Athletes: A Case Study in Rugby Union.

Practical case studies elaborating end-to-end attempts to improve the quality of information flows associated with athlete management processes are scarce in the current sport literature. Therefore, guided by a Business Process Management (BPM) approach, the current study presents the outcomes from a case study to optimize the quality of strength and conditioning (S&C) information flow in the performance department of a professional rugby union club. Initially, the S&C information flow was redesigned using integral technology, activity elimination and activity automation redesign heuristics. Utilizing the Lean Startup framework, the redesigned information flow was digitally transformed by designing data collection, management and visualization systems. Statistical tests used to assess the usability of the data collection systems against industry benchmarks using the System Usability Scale (SUS) administered to 55 players highlighted that its usability (mean SUS score of 87.6 ± 10.76) was well above average industry benchmarks of similar systems (Grade A from SUS scale). In the data visualization system, 14 minor usability problems were identified from 9 cognitive walkthroughs conducted with the High-Performance Unit (HPU) staff. Pre-post optimization information quality was subjectively assessed by administering a standardized questionnaire to the HPU members. The results indicated positive improvements in all of the information quality dimensions (with major improvements to the accessibility) relating to the S&C information flow. Additionally, the methods utilized in the study would be especially beneficial for sporting environments requiring cost effective and easily adoptable information flow digitization initiatives which need to be implemented by its internal staff members.

Digitally Mediated Schizophrenia Care - A Swedish Case of Translating, Designing and Expert Evaluation of Dialog.

This work presents initial results of translation, design, development and expert testing of a digitally mediated dialogue based tool for schizophrenia care in Sweden. Dialog+ is a conversational methodology that has been shown to empower patients, promote co-produced and more cost-effective care. Dialog+ was translated and the digital application was redesigned to fit with the local context using human-centered design principles. Initial results of expert user testing, using heuristic evaluations and cognitive walkthroughs, shows promise for the method to work well also in Swedish settings. Initial testing with patients shows promising results. This work exemplifies how care patterns can be improved by considering the information layer of the interaction and creating shared and collaborative working spaces during treatment sessions can help to both empower patients and facilitate a more co-produced treatment plan.

SYSTEMATIC REVIEW OF THE USABILITY EVALUATION METHODOLOGY OF HEALTH INFORMATION SYSTEMS

A good information system will support the implementation of the tasks and functions of the organization. This systematic review aims to identify and characterize the methods and attributes in usability evaluation and identify measurement techniques in usability evaluation. This systematic review uses the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology. The search was carried out using two databases, namely PubMed and Science Direct, by limiting information systems related to health published from 2015 to 2021. Articles that met the criteria were evaluated for methods, attributes, measurement techniques, and usability criteria used. Article quality was assessed using a modified Downs and Black checklist. The search identified 132 articles. The results of abstract screening and duplication-checking obtained 128 articles. The screening was carried out through a complete review of the inclusion and exclusion criteria, obtained 9 articles that met the criteria. There are five methods used for usability evaluation, namely Think aloud (TA), Heuristic evaluation (HE), System Usability Scale (SUS), User viewpoint, and Cognitive Walkthrough Method. Six problem definitions used in usability evaluation are ISO, Nielsen, user-based, TAUCHI, 19 item usability questions, and successful completion of scenarios. The measurement technique is done by qualitative and quantitative analysis. Effectiveness, efficiency, and satisfaction are criteria that are always assessed from almost all articles reviewed. The heuristic evaluation method with quantitative measurement techniques using an evaluator in evaluating usability based on scenarios then proceeding with qualitative measurement techniques to deepen problems and provide solutions is the most ideal usability evaluation measurement technique.

Development of a mental script for the mental practice of micro suturing: a methodological approach.

Motor imagery and mental practice are important for the acquisition and mastery of surgical skills. The success of this technique relies on the use of a well-developed mental script. In this study, we shared how we developed a mental script for basic micro suturing training by using a low-fidelity rubber glove model. This study applied the design and development research framework. Five expert surgeons developed a mental script by performing a cognitive walkthrough to repair a vertical opening in a rubber glove model, followed by hierarchical task analysis. A draft script was created, and its face and content validity assessed with a checking-back process. Twenty-eight surgeons used the Mental Imagery Questionnaire (MIQ) to assess the validity of the final script. The process of developing the mental script is detailed. The assessment by the expert panel showed the mental script had good face and content validity. The mean overall MIQ score was 5.2±1.1 (standard deviation), demonstrating the validity of generating mental imagery from the mental script developed in this study for micro suturing in the rubber glove model. The methodological approach described in this study is based on a design and development research framework to teach surgical skills. This model is inexpensive and easily accessible, addressing the challenges of reduced opportunities to practise surgical skills. However, although motor skills are important, the surgeon's other non-technical expertise is not addressed with this model. Thus, this model should act as one surgical training approach, but not replace it.

Perancangan Tampilan Aplikasi Pembelajaran Pinandita dengan Metode Five Planes, Heuristic Evaluation, Concurrent Think Aloud, Serta Cognitive Walkthrough

The design and development of the application display uses five areas to find out more about the needs and requirements of the user, so that it will be easy to learn and understand by the user himself. The research variables used in determining the ease of use of the application are the effectiveness and efficiency of using the application display and user satisfaction with the application display used. The sampling technique used here is purposive sampling, so the total sample used is 30 people. The research sample used is divided into 2 parts, namely, the general public and people who have an information technology educational background. The data in the study were collected by conducting interviews and questionnaires for the testing phase. Each method used is representative of the research variables. The test method used in this study is a cognitive walkthrough representing the effectiveness variable, heuristic evaluation representing the efficiency variable, and concurrent think aloud representing the user satisfaction variable.

Web-Based Tool (FH Family Share) to Increase Uptake of Cascade Testing for Familial Hypercholesterolemia: Development and Evaluation.

BackgroundFamilial hypercholesterolemia, a prevalent genetic disorder, remains significantly underdiagnosed in the United States. Cascade testing, wherein individuals diagnosed with familial hypercholesterolemia— probands—contact their family members to inform them of their risk for familial hypercholesterolemia, has low uptake in the United States. Digital tools are needed to facilitate communication between familial hypercholesterolemia probands and their family members and to promote sharing of familial hypercholesterolemia–related risk information.ObjectiveWe aimed to create and evaluate a web-based tool designed to enhance familial communication and promote cascade testing for familial hypercholesterolemia.MethodsA hybrid type 1 implementation science framework and a user-centered design process were used to develop an interactive web-based tool—FH Family Share—that enables familial hypercholesterolemia probands to communicate information about their familial hypercholesterolemia diagnosis with at-risk relatives. Probands can also use the tool to draw a family pedigree and learn more about familial hypercholesterolemia through education modules and curated knowledge resources. Usability guidelines and standards were taken into account during the design and development of the tool. The initial prototype underwent a cognitive walkthrough, which was followed by usability testing with key stakeholders including genetic counselors and patients with familial hypercholesterolemia. Participants navigated the prototype using the think-aloud technique, and their feedback was used to refine features of the tool.ResultsKey themes that emerged from the cognitive walkthrough were design, format, navigation, terminology, instructions, and learnability. Expert feedback from the cognitive walkthrough resulted in a rebuild of the web-based tool to align it with institutional standards. Usability testing with genetic counselors and patients with familial hypercholesterolemia provided insights on user experience, satisfaction and interface design and highlighted specific modifications that were made to refine the features of FH Family Share. Genetic counselors and patients with familial hypercholesterolemia suggested inclusion of the following features in the web-based tool: (1) a letter-to-family-member email template, (2) education modules, and (3) knowledge resources. Surveys revealed that 6 of 9 (67%) genetic counselors found information within FH Family Share very easy to find, and 5 of 9 (56%) genetic counselors found information very easy to understand; 5 of 9 (56%) patients found information very easy to find within the website, and 7 of 9 (78%) patients found information very easy to understand. All genetic counselors and patients indicated that FH Family Share was a resource worth returning to.ConclusionsFH Family Share facilitates communication between probands and their relatives. Once informed, at-risk family members have the option to seek testing and treatment for familial hypercholesterolemia.

User Perceptions of Security and Privacy for Group Chat

Secure messaging tools are an integral part of modern society. To understand users’ security and privacy perceptions and requirements for secure group chat, we surveyed 996 respondents in the US and UK. Our results show that group chat presents important security and privacy challenges, some of which are not present in one-to-one chat. For example, users need to be able to manage and monitor group membership, establish trust for new group members, and filter content that they share in different chat contexts. We also find that respondents lack mechanisms for determining which tools are secure and instead rely on non-technical strategies for protecting their privacy—for example, self-filtering and carefully tracking group membership. To better understand how these results relate to existing tools, we conduct cognitive walkthroughs (a form of expert usability review) for five popular group chat tools. Our results demonstrate that while existing tools address some items identified in our surveys, this support is partial and is insufficient in many cases. As such, there is a need for improved group chat tools that better align with user perceptions and requirements. Based on these findings, we provide recommendations on improving the security and usability of secure group chat.

Semantic knowledge modeling and evaluation of argument Theory to develop dialogue based patient education systems for chronic disease Self-Management

BackgroundTo improve understanding as well as uptake of health educational material, it should be tailored to informational needs, offer intuitive modes of interaction, and present credible evidence for health claims. Dialogue systems go some way in meeting these requirements, as they emulate interactive and intuitive person-to-person communication. However, most works do not offer a formal model nor modelling process to structure dialogue content, and do not focus on ensuring credibility. MethodsWe propose an Extended Model of Argument (EMA) dialogue model and modelling process to support educational dialogue systems. In this dialogue model, computerized arguments directly offer evidence for health claims. EMA further offers “dialogue by design”, where argument structures and interrelations are dynamically leveraged to offer dialogues, instead of relying on predefining discourse flows. We implemented an EMA-based dialogue education system for Juvenile Idiopathic Arthritis (JIA). We performed a qualitative evaluation with JIA health experts involving a Cognitive Walkthrough and Semi-Structured Interview. We applied Directed Content Analysis using categories from the O’Grady framework, and coded sub-themes within those categories using Grounded Theory. ResultsWe identified 6 sub-themes within the participant feedback pertaining to Quality, Credibility, and Utility. Participants attached strong importance to credibility and found the dialogue system to be a flexible educational tool. Some participants suggested sorting educational items by importance, and presenting only salient knowledge associations to reduce dialogue complexity. ConclusionOverall, our qualitative evaluation confirmed the following: the ability of EMA to offer credible and appropriate dialogues; and, in general, the utility of dialogue systems to educate JIA patients and their families. In future work, we will revise the system based on evaluation feedback, and perform a more extensive evaluation with patients and caregivers.

Usability Inspection of a Mobile Cancer Telerehabilitation System.

Cancer rehabilitation (CR) has been shown to address physical impairments and progressive disablement experienced by patients with cancer. Access to CR is limited by multiple barriers. Previous studies demonstrated that telemedicine approaches may facilitate access to rehabilitation services however usability and acceptance of cancer telerehabilitation has not been systematically evaluated. This goal of this study was usability inspection of a mobile cancer telerehabilitation system based on cognitive walkthroughs and heuristics evaluations, supplemented by surveys to capture health literacy, patient engagement, and acceptance. The System Usability Scale provided a standardized reference for usability and satisfaction, and the mean result of 83.2±16.9 placed this mobile interface in the top 10th percentile. A semi-structured qualitative interview provided actionable feedback, which informed the next iteration of this project. Overall, this mobile telerehabilitation system was considered by cancer patients easy-to-use, satisfying, and engaging with 91% of participants planning to use it in the future.

Clinical decision support for intervention reduction in neonatal patients: A usability assessment.

ObjectiveThis study investigated how effectively simplified cognitive walkthroughs, performed independently by four nonclinical researchers, can be used to assess the usability of clinical decision support software. It also helped illuminate the types of usability issues in clinical decision support software tools that cognitive walkthroughs can identify.MethodA human factors professor and three research assistants each conducted an independent cognitive walkthrough of a web-based demonstration version of T3, a physiologic monitoring system featuring a new clinical decision support software tool called MAnagement Application (MAP). They accessed the demo on personal computers in their homes and used it to walk through several pre-specified tasks, answering three standard questions at each step. Then they met to review and prioritize the findings.ResultsEvaluators acknowledged several positive features including concise, helpful tooltips and an informative column in the patient overview which allows users direct (one-click) access to protocol eligibility and compliance criteria. Recommendations to improve usability include: modify the language to clarify what user actions are possible; visually indicate when eligibility flags are snoozed; and specify which protocol's data is currently being shown.ConclusionIndependent, simplified cognitive walkthroughs can help ensure that clinical decision support software tools will appropriately support clinicians. Four researchers used this technique to quickly, inexpensively, and effectively assess T3's new MAP tool, which suggests positive actions, such as removing a patient from a ventilator. Results indicate that, while there is room for usability improvements, the MAP tool may help reduce clinician's cognitive load, facilitating improved care. The study also confirmed that cognitive walkthroughs identify issues that make clinical decision support software hard to learn or remember to use.

Design and Evaluation of a User Interface for an Autonomous Agricultural Sprayer

This paper describes the design and evaluation of a user interface for a remotely supervised autonomous agricultural sprayer. The interface was designed to help the remote supervisor to instruct the autonomous sprayer to commence operation, monitor the status of the sprayer and its operation in the field, and intervene when needed (i.e., to stop or shut down). Design principles and guidelines were carefully selected to help develop a human-centered automation interface. Evaluation of the interface using a combination of heuristic, cognitive walkthrough, and user testing techniques revealed several strengths of the design as well as areas that needed further improvement. Overall, this paper provides guidelines that will assist other researchers to develop an ergonomic user interface for a fully autonomous agricultural machine.

A web-based comprehensive head and neck cancer patient education and support needs program: Usability testing.

Background: With increasing accessibility of the World Wide Web, patients are using it to obtain patient education materials (PEM). With this in mind, our group (surgeons, radiation oncologists, medical oncologists, nursing, allied health professionals and academic researchers) developed a comprehensive information resource for patients with head and neck cancer (HNC), the Head and Neck Cancer Australia, formerly Beyond Five website. The aim of this study was to determine patient usability of the Head and Neck Cancer Australia website. Methods: Usability testing (Cognitive walkthrough & Think-Aloud) in 18 patients treated for HNC was undertaken at a Cancer centre (anonymous for reviewers). Results: The tasks rated easiest by patients were finding information on psychological well-being (mean time spent: 66s, range: 10-565), health and well-being effecting quality of life (mean time spent: 36s, range 9-117) and carer information (mean time spent: 10s, range 3-35). Patients indicated the website contained a lot of information, covers most topics, was a trusted source of information and a springboard to other information. Conclusions: The Head and Neck Cancer Australia website provides a wide range of information and support in multiple formats available to HNC patients. Further refinements in design, navigation and website instructions are needed to allow effective patient interaction.

TASK CENTERED SYSTEM DESIGN PADA PERANCANGAN APLIKASI MOBILE SISTEM INFORMASI AKADEMIK (Studi Kasus: UPTTIK Universitas Siliwangi)

This research was conducted to provide a User Interface recommendation on the SIMAK Mobile application. This study entitled "TASK CENTERED SYSTEM DESIGN IN DESIGN OF ACADEMIC INFORMATION SYSTEM MOBILE APPLICATIONS (Case Study: UPTTIK Siliwangi University)", uses the Cognitive walkthrough method to test the results of user interface design. Was retested using the System Usability Scale method. The results of the System Usability Scale test show a score of 88, 75 or in the acceptable category.

Ambient Intelligence System Enabling People With Blindness to Develop Electrotechnical Components and Their Drivers

Along with the development of new assistive technologies, including ambient intelligence (AmI) environments, new job opportunities are created for people with blindness or visual impairment. Whereas research on software development for people with blindness has been conducted since the 1960s, the design and development of electrotechnical equipment still lacks any way to compensate for the disability which would enable people with blindness to perform even the activities pertaining to this field. This article aims to introduce these new technological procedures enabling the compensation of the disability of people with blindness when designing and developing electrotechnical components (or prototypes) and their drivers by using the AmI system RUDO modules and other tools. This includes the modules enabling measuring electrotechnical quantities, tracing of oscilloscope curves, and ensuring a unified user interface for programming and operation of the devices connected. Other approaches are introduced herein that focused on the production of mechanical components for device construction. This article also contains an illustrative video of practical use of these technological procedures by a person with blindness. To assess the usability of the designed technological procedures to the work of a designer with blindness and one without visual impairment, the cognitive walkthrough method was used. The main contribution of this article is to broaden the knowledge base by the principles of involving people with blindness in the development and construction of electrotechnical components (or prototypes). These new possibilities can be used practically, for instance, in computer education, which can offer new curricula for visually impaired students and focused on more practical issues where hardware and software approaches meet.

Evaluasi Kebergunaan (Usability) dan Rekomendasi Penggunaan Google Classroom untuk Blended Learning di Perguruan Tinggi

The google classroom application is a virtual classroom mixed with real classes in education where the distribution of material and lecture assignments can be done by teachers and students without using paper or paperless. Interaction between teachers and students is done interactively online. The use of the google classroom application as a teaching tool has only been implemented at XYZ college, so usability testing is needed in the use of google classroom. Testing the usability level of Google Classroom is really needed to evaluate and get recommendations about using the Google Classroom application. The method used in this study is the USE (Usefullness, Satisfaction and Ease of Use) Questionnaire and Cognitive Walkthrough (CW) used in the google classroom usability test. USE stands for Usefulness, Satisfaction, and Ease of Use. Ease of use factors can be divided into two, namely Ease of Learning and Ease of Use. The USE test involved all XYZ college students, totaling 213 respondents. Ten respondents within certain criteria were selected by purposive sampling technique to test the usefulness of using the cognitive walkthrough (CW) method. The overall assessment of the usefulness factor with a mean value of 3.95 is included in the "Good" rating range. Overall, the satisfaction factor value of 4.01 is included in the "Good" rating range. Overall, the assessment of the Ease of use indicator with a mean of 4.22 is included in the "Good" rating range. The cognitive walkthrough (CW) method produces several recommendations that can be utilized by XYZ universities in implementing blended learning using google classroom. The usability factor with various features offered by Google Classroom is quite high because it can provide convenience in its use to support lecture activities at XYZ College

A comparative analysis of cryptocurrency wallet management tools

The work is devoted to a comparative analysis of tools for managing a cryptocurrency portfolio. The study aimed to find out which of the tools is currently the best solution for users. This analysis was carried out on the basis of two tools, first: a survey conducted of among 41 responders who are cryptocurrency users, and the second one: cognitive walkthrough. Conducted analysis confirmed the thesis that the Trust Wallet tool is currently the best solution for users.

Cognitive Testing of an Electronic Consent Platform: Researcher Perspectives.

This study focuses on feedback from domain experts to assess usability and acceptance of the E-Consent electronic consent platform. Quantitative and qualitative data were captured throughout the usability inspection, which was structured around a cognitive walkthrough with heuristics evaluation. Additional surveys measured biobanking knowledge and attitudes and familiarity with informed consent. A semi-structured qualitative interview captured open-ended feedback. 23 researchers of various ages and job titles were included for analysis. The System Usability Scale (SUS) provided a standardized reference for usability and satisfaction, and the mean result of 86.7 corresponds with an 'above average' usability rating in the >90th percentile. Overall, participants believe that electronic consenting using this platform will be faster than previous workflows while enhancing patient understanding, and human rapport is still a key component of the consent process. Expert review has provided valuable insight and actionable information that will be used to further enhance this maturing platform.