Mean System Usability Scale Score Research Articles

Usability of an AI-based independent pedestrian navigation device designed for blind people

Abstract Background 2.2 billion people live with visual impairment (VI) worldwide and face orientation and mobility challenges in various aspects of their lives. Despite technological advancements, the most used mobility solutions for VI are still the walking cane and the guide dog. The objective of this research is to assess the system usability of an AI-based independent pedestrian navigation device (.lumen glasses), designed for blind people. Methods During 31st of October and 2nd of November 2023, 27 visually impaired people living in Bucharest, Romania, tested the .lumen glasses for 1-3 hours during an outdoor navigation task. The glasses use AI to recognize the user’s environment and haptic and audio feedback to guide them. The System Usability Scale (SUS), a 10-item instrument, was used to evaluate the level of usability the participants ascribe to the .lumen glasses. Qualitative feedback was collected as part of the post-test survey and analyzed using thematic analysis. Data were triangulated to identify usability issues. Results The mean age of the participants was 30.3 years old (SD = 12.07, min=21 max =56, range: 48), 55.6% (N = 15) of the participants were male, 96,3% (N = 26) were completely blind, and 81.5% were using a white cane for orientation and mobility. Our findings reveal a mean SUS score of 83.65 (SD = 15.23, range 47.5, min=52.5, max=100) indicating high usability. The thematic analysis revealed 5 themes: (1) auditive and (2) haptic feedback in navigation, (3) hardware (adjustment system & buttons), (4) tutorial & presentation of how the device works and (5) other aspects (usage in combination with other assistive technologies etc.). The participants declared that the glasses are intuitive to use and promote the feeling of safety and independence. Conclusions Data suggest that the .lumen glasses area promising assistive technology for the mobility of VI. Findings are currently used to steer the development and implementation of the AI-based system. Key messages • According to participants’ feedback, .lumen glasses are a scalable AI-based mobility solution that can make navigation and orientation easier for visually impaired individuals. • .lumen glasses designed for the visually impaired individuals are one of the current and limited assistive technology improving the mobility and orientation.

Development and Pilot Study of myfood24 West Africa—An Online Tool for Dietary Assessment in Nigeria

Background and objective: Tools to accurately and efficiently measure dietary intake in Nigeria are lacking. We aimed to develop and assess the usability of a new online dietary assessment tool for Nigeria—myfood24 West Africa. Methods: We developed the myfood24 West Africa database using data from existing food composition tables, packaged foods labels and research articles. The development followed seven steps: identified data sources, selected foods, processed/cleaned the data, calculated the nutrient content of recipes, created and allocated portion sizes, quality-checked the database and developed food accompaniments. To pilot the tool, we recruited 179 university staff in Nigeria using a cross-sectional design. Usability was assessed using a questionnaire that included the System Usability Scale (SUS) and a feedback session. Results: The database included 924 foods, with up to 54 nutrients and 35 portion-size images allocated to foods. Sixty percent of the data were sourced from the 2019 West Africa Food Composition Table, 17% from back-of-pack labels of packaged foods, 14% from the 2017 Nigerian Food Composition Table, 5% from generated recipes and 4% from the published literature. Of the participants, 30% (n = 53) self-recorded their food intake, with a total of 1345 food and drink entries from both self- and interviewer-collected data. The mean SUS score of 74 (95% CI: 68,79) indicated good usability. The feedback showed that the tool was user-friendly, educational and included a variety of local foods. Conclusions: This new tool will enhance the dietary assessment of the Nigerian population. More work will expand coverage to include more foods from the region.

Interactive Serious Game to Teach Basic Life Support Among Schoolchildren in Brazil: Design and Rationale.

Cardiovascular diseases are leading causes of death and morbidity worldwide. CPR and early defibrillation significantly enhance survival rates. Serious Games (SG) improve learning through entertainment. Current strategies target Cardiopulmonary resuscitation for communities and schoolchildren, but none have been validated for children in low-to-middle-income settings. The SG Children Save Hearts, developed in Brazil, teaches the five resuscitation steps according to International Liaison Committee on Resuscitation 2020 guidelines and requires formal usability assessment. The study aimed to evaluate the usability of SG Children Save Hearts among IT and healthcare professionals using the System Usability Scale (SUS). The usability test was conducted in August 2022 in the university's IT department. The game was developed targeting schoolchildren aged 7 to 17. Categorical variables as absolute and relative frequencies, while continuous variables were presented as median with interquartile range (IQR). Normality was assessed using the Shapiro-Wilk test. Comparisons between IT and healthcare professionals were made using the independent t-test for normal distributions or the Mann-Whitney U test for non-normal distributions. We included 17 volunteers with a mean age of 22 years (IQR 20-26). All participants played the game and completed a 10-question survey on its usability using a Likert-type scale. The final grade was converted to a 0 to 100 scale, with a grade above 70 considered acceptable for a minimum viable product. The mean SUS score was 75 (IQR 72.5-87.5). Healthcare professionals gave higher grades to all five domains compared to IT professionals. The average time spent playing the game was 3.2 minutes. Novel technologies have shown promising results for CPR teaching using active teaching methods, but face challenges in developing countries, such as language barriers, device acquisition, cultural differences, and technical support. To our knowledge, this is the first SG developed in portuguese for brazilian schoolchidren. Despite some usability issues, the SG Children Save Hearts is considered adequate for teaching CPR to schoolchildren in Brazil.

Usability, Engagement, and Report Usefulness of Chatbot-Based Family Health History Data Collection: Mixed Methods Analysis.

Family health history (FHx) is an important predictor of a person's genetic risk but is not collected by many adults in the United States. This study aims to test and compare the usability, engagement, and report usefulness of 2 web-based methods to collect FHx. This mixed methods study compared FHx data collection using a flow-based chatbot (KIT; the curious interactive test) and a form-based method. KIT's design was optimized to reduce user burden. We recruited and randomized individuals from 2 crowdsourced platforms to 1 of the 2 FHx methods. All participants were asked to complete a questionnaire to assess the method's usability, the usefulness of a report summarizing their experience, user-desired chatbot enhancements, and general user experience. Engagement was studied using log data collected by the methods. We used qualitative findings from analyzing free-text comments to supplement the primary quantitative results. Participants randomized to KIT reported higher usability than those randomized to the form, with a mean System Usability Scale score of 80.2 versus 61.9 (P<.001), respectively. The engagement analysis reflected design differences in the onboarding process. KIT users spent less time entering FHx information and reported more conditions than form users (mean 5.90 vs 7.97 min; P=.04; and mean 7.8 vs 10.1 conditions; P=.04). Both KIT and form users somewhat agreed that the report was useful (Likert scale ratings of 4.08 and 4.29, respectively). Among desired enhancements, personalization was the highest-rated feature (188/205, 91.7% rated medium- to high-priority). Qualitative analyses revealed positive and negative characteristics of both KIT and the form-based method. Among respondents randomized to KIT, most indicated it was easy to use and navigate and that they could respond to and understand user prompts. Negative comments addressed KIT's personality, conversational pace, and ability to manage errors. For KIT and form respondents, qualitative results revealed common themes, including a desire for more information about conditions and a mutual appreciation for the multiple-choice button response format. Respondents also said they wanted to report health information beyond KIT's prompts (eg, personal health history) and for KIT to provide more personalized responses. We showed that KIT provided a usable way to collect FHx. We also identified design considerations to improve chatbot-based FHx data collection: First, the final report summarizing the FHx collection experience should be enhanced to provide more value for patients. Second, the onboarding chatbot prompt may impact data quality and should be carefully considered. Finally, we highlighted several areas that could be improved by moving from a flow-based chatbot to a large language model implementation strategy.

Usability of an at-home tablet-based cognitive test in older adults with and without cognitive impairment

BackgroundMobile device-based cognitive screening has the potential to overcome the limitations in diagnostic precision and efficiency that characterize conventional pen and paper cognitive screening. Several mobile device-based cognitive testing platforms have demonstrated usability, but the usability of take-home mobile device-based cognitive screening in typical adult primary care patients requires further investigation.MethodsThis study set out to test the usability of a prototype mobile device-based cognitive screening test in older adult primary care patients across a range of cognitive performance. Participants completed the St. Louis University Mental Status Examination (SLUMS) and then used a study-supplied mobile device application at home for 5 days. The application presented 7 modules lasting approximately 15 min. Participants completed the System Usability Scale (SUS) after using the application.ResultsA total of 51 individuals participated, with a median (IQR) age of 81 (74–85) years. Cognitive impairment (SLUMS score < 27) was present in 30 (59%) of participants. The mean (95% Confidence Interval [CI]) SUS score was 76 (71–81), which indicates good usability. Usability scores were similar across ranges of cognitive impairment. A Lower SLUMS score predicted early withdrawal from the study with an area under the receiver operating characteristic curve (95% CI) of 0.78 (0.58–0.97).ConclusionTake-home mobile device-based cognitive testing is a usable strategy for many older adult primary care patients. Depending on patient preferences and abilities, it could be part of a flexible cognitive testing and follow-up strategy that includes mobile device-based testing in healthcare settings and pen-and-paper cognitive testing.

Early feasibility and usability study of a novel obstetric blood loss quantifying device

ObjectiveTo study the accuracy and usability of a novel obstetric blood loss quantifying tool in clinical settings. MethodsA mixed-methods study was conducted in an Irish tertiary maternity unit. The accuracy of measuring the blood content (hemoglobin concentration) of elective Caesarean section birth waste with a novel obstetric blood loss quantifying device was compared, using Bland-Altman and correlation analysis, with staff volumetry and a reference hemoglobinometer. Hospital staff (nurses, midwives and doctors) opinion of the usability of the device was studied by an anonymous questionnaire which generated a System Usability Scale (SUS) score. SUS scores range from 0 to 100 with higher scores reflecting greater usability. ResultsThe device was used by 19 different hospital staff members (nine nurses, four midwives and six doctors) in 19 elective Caesarean deliveries that had varying levels of PPH risk. Bland-Altman analysis produced mean biases of −0.7 ± 1.5 g/dL and 0.2 ± 1.2 g/dL for the device and staff measurements respectively. The width of the limits of agreement was narrower for device measurements than for staff measurements (4.5 g/dL and 5.7 g/dL respectively). The device’s measurements of hemoglobin content correlated more strongly with the hemoglobinometer rather than with hospital staff measurements (Spearman correlation coefficients of 0.9 and 0.6 respectively). This suggests that the device is more accurate in determining the blood content of the birth waste than hospital staff volumetric measurements. Hospital staff members assigned the device a mean SUS score of 82 which suggests that the device is highly usable. ConclusionAn early feasibility study in clinical settings suggests that a novel device for quantifying obstetric blood loss was more accurate than volumetry in measuring the hemoglobin content of birth waste. Health professionals also found the device highly usable. This data suggests that there is much potential in transitioning from a “human-made” to a “machine-made” assessment of blood loss. Future studies will entail additional testing of the device to assess its impact on the morbidities associated with postpartum hemorrhage.

A Remote Patient Monitoring System with Feedback Mechanisms using a Smartwatch: Concept, Implementation and Evaluation based on the activeDCM Randomized Controlled Trial.

Technological advances allow recording and sharing of health-related data in a patient-centric way using smartphones and wearables. Secure sharing of such patient-generated data with physicians would enable a dense management of individual health trajectories, monitoring of risk factors and asynchronous feedback. However, most Remote Patient Monitoring (RPM) systems currently available are not fully integrated into hospital IT systems or lack the patient-centric design. The objective was to conceptualize and implement a user-friendly, reusable, interoperable and secure RPM system incorporating asynchronous feedback mechanisms, using a broadly available consumer wearable (Apple Watch). Additionally, the study sought to evaluate factors influencing patient acceptance of such systems. The RPM system requirements were established through focus group sessions. Subsequently, a system concept was designed and implemented using an iterative approach, ensuring technical feasibility from the beginning. To assess clinical feasibility, the system was employed as part of the activeDCM prospective, randomized, interventional study focusing on Dilated Cardiomyopathy (DCM). Each patient used the system for at least 12 months. The System Usability Scale (SUS) was employed to measure usability from a subjective patient perspective. Additionally, an evaluation was conducted on the objective wearable interaction frequency as well as the completeness of transmitted data, classified into Sensor-based Health Data (SHD) and Patient Reported Outcome Measures (PROM). Descriptive statistics using boxplots, along bootstrapped multiple linear regression with a 95% confidence interval (CI) were utilized for evaluation, analyzing the influence of age, sex, device experience and intervention group membership. The RPM system consists of four interoperable components: patient-devices, data-server, data-viewer and notification-service. The evaluation of the system was conducted with 95 consecutive DCM patients (female: 28 of 95 (29%), age: 50±12 years) completing the activeDCM study protocol. The wearable/ smartphone application of the system achieved a mean SUS score of 78±17, which was most influenced by device experience. 83 of 95 patients (87%) could integrate the wearable application (very) well into their daily routine and 67 of 95 (70%) saw a benefit of the RPM system for management of their health condition. Patients interacted on average with the wearable on 61%±26% of days enrolled in the study, corresponding to 239±99 of 396±39 days. SHD was available on average for 78%±23% of days and PROM data 64%±27% of weeks enrolled in the study, corresponding to 307±87 of 396±39 days and 35±15 of 56±5 weeks, respectively. Wearable interaction frequency, SHD and PROM completeness were most influenced by intervention group membership. Our results mark a first step towards integrating RPM systems, based on a consumer wearable device for primary patient input, into standardized clinical workflows. They can serve as a blueprint for creating a user-friendly, reusable, interoperable and secure RPM system, that can be integrated into patients' daily routines. ClinicalTrials.gov-Identifier: NCT04359238.

Design and usability evaluation of a mobile app for elementary school inquiry‐based science learning

AbstractTeachers often struggle to implement inquiry‐based science teaching. To support them, IndagApp—a 3D educational app that offers curriculum‐aligned, inquiry‐based lesson plans—was designed. The app is rooted in the inquiry phases recommended in best‐practices literature, which arguably align with most international standards. This study describes IndagApp and evaluates its usability with fifth‐graders from two elementary schools in Spain (N = 43). The system usability scale (SUS) and the Pictorial‐SUS were used to collect students' feedback on usability. Both frequentist and Bayesian analyses were conducted to compare the mean SUS score with established benchmarks for usability. The results showed that IndagApp had high usability ratings, with most students rating it as the “Best imaginable” or “Good.” The mean SUS score was 84.816, which was significantly higher than the benchmark score of 68. There were no differences in usability between girls and boys, and students in private and public schools. These findings suggest that IndagApp is a valuable resource for inquiry learning in elementary grades and has significant implications for science education and science teacher professional development, as it helps teachers adopt reform‐oriented teaching practices that align with the curricular standards and goals.

Usability and Preliminary Efficacy of an Adaptive Supportive Care System for Patients With Cancer: Pilot Randomized Controlled Trial.

Using an iterative user-centered design process, our team developed a patient-centered adaptive supportive care system, PatientCareAnywhere, that provides comprehensive biopsychosocial screening and supportive cancer care to patients across the continuum of care adaptively. The overarching goal of PatientCareAnywhere is to improve health-related quality of life (HRQOL) and self-efficacy of patients with cancer by empowering them with self-management skills and bringing cancer care support directly to them at home. Such support is adaptive to the patient's needs and health status and coordinated across multiple sources in the forms of referrals, education, engagement of community resources, and secure social communication. This study aims to assess the usability of the new web-based PatientCareAnywhere system and examine the preliminary efficacy of PatientCareAnywhere to improve patient-reported outcomes compared with usual care. For phase 1, usability testing participants included patients with cancer (n=4) and caregivers (n=7) who evaluated the software prototype and provided qualitative (eg, interviews) and quantitative (eg, System Usability Scale) feedback. For phase 2, participants in the 3-month pilot randomized controlled trial were randomized to receive the PatientCareAnywhere intervention (n=36) or usual care control condition (n=36). HRQOL and cancer-relevant self-efficacy were assessed at baseline (preintervention assessment) and 12 weeks from baseline (postintervention assessment); mean differences between pre- and postintervention scores were compared between the 2 groups. Participants were highly satisfied with the prototype and reported above-average acceptable usability, with a mean System Usability Scale score of 84.09 (SD 10.02). Qualitative data supported the overall usability and perceived usefulness of the intervention, with a few design features (eg, "help request" function) added based on participant feedback. With regard to the randomized controlled trial, patients in the intervention group reported significant improvements in HRQOL from pre- to postintervention scores (mean difference 6.08, SD 15.26) compared with the control group (mean difference -2.95, SD 10.63; P=.01). In contrast, there was no significant between-group difference in self-efficacy (P=.09). Overall, PatientCareAnywhere represents a user-friendly, functional, and acceptable supportive care intervention with preliminary efficacy to improve HRQOL among patients diagnosed with cancer. Future studies are needed to further establish the efficacy of PatientCareAnywhere as well as explore strategies to enhance user engagement and investigate the optimal intensity, frequency, and use of the intervention to improve patient outcomes. ClinicalTrials.gov NCT02408406; https://clinicaltrials.gov/study/NCT02408406.

Interactive Health Technology Tool for Kidney Living Donor Assessment to Standardize the Informed Consent Process: Usability and Qualitative Content Analysis.

Kidney living donation carries risks, yet standardized information provision regarding nephrectomy risks and psychological impacts for candidates remains lacking. This study assesses the benefit of interactive health technology in improving the informed consent process for kidney living donation. The Kidney Hub institutional open portal offers comprehensive information on kidney disease and donation. Individuals willing to start the kidney living donation process at Helsinki University Hospital (January 2019-January 2022) were invited to use the patient-tailored digital care path (Living Donor Digital Care Path) included in the Kidney Hub. This platform provides detailed donation process information and facilitates communication between health care professionals and patients. eHealth literacy was evaluated via the eHealth Literacy Scale (eHEALS), usability with the System Usability Scale (SUS), and system utility through Likert-scale surveys with scores of 1-5. Qualitative content analysis addressed an open-ended question. The Kidney Hub portal received over 8000 monthly visits, including to its sections on donation benefits (n=1629 views) and impact on donors' lives (n=4850 views). Of 127 living kidney donation candidates, 7 did not use Living Donor Digital Care Path. Users' ages ranged from 20 to 79 years, and they exchanged over 3500 messages. A total of 74 living donor candidates participated in the survey. Female candidates more commonly searched the internet about kidney donation (n=79 female candidates vs n=48 male candidates; P=.04). The mean eHEALS score correlated with internet use for health decisions (r=0.45; P<.001) and its importance (r=0.40; P=.01). Participants found that the Living Donor Digital Care Path was technically satisfactory (mean SUS score 4.4, SD 0.54) and useful but not pivotal in donation decision-making. Concerns focused on postsurgery coping for donors and recipients. Telemedicine effectively educates living kidney donor candidates on the donation process. The Living Donor Digital Care Path serves as a valuable eHealth tool, aiding clinicians in standardizing steps toward informed consent. ClinicalTrials.gov NCT04791670; https://clinicaltrials.gov/study/NCT04791670. RR2-10.1136/bmjopen-2021-051166.

P-601 Monitoring reproductive hormones during ovarian stimulation: performance and usability of a self-operated at-home urine test

Abstract Study question How effective and user-friendly is a non-invasive, patient-operated urine test (Kinder) in tracking reproductive hormone levels during ovarian stimulation (OS) treatment in an IVF cycle? Summary answer Urinary E1-3G correlates well with blood E2 levels and ovarian follicle development. Patients found it easy and stress-free to perform the urine test at home. What is known already Monitoring blood reproductive hormone levels during OS is a widely accepted practice to ensure the optimal ovarian response to gonadotropin treatment, timing of oocyte maturation triggering and mitigating hyperstimulation risk. However, conventional blood testing is invasive, necessitating patients to travel to clinics for blood sampling, adding to their overall burden. Previous studies have revealed a robust correlation between blood E2 levels, and its corresponding urinary metabolites E1-3G. Kinder is a portable device designed for quantifying urinary E1-3G, allowing patients to conduct tests in the privacy of their homes, and for many obviating the need to travel substantial distances for monitoring. Study design, size, duration This is a prospective, single-cohort, study of Caucasian patients (28-34 years), conducted at Fertility Specialists of Western Australia in Perth, Australia, where patients may travel vast distances for treatment. Twenty-five patients were enrolled from Aug 2022-Aug 2023, and performed home urine tests utilizing the Kinder device in parallel with clinic scheduled blood (ELISA) and ultrasound tests. The primary end point of the study was the correlation between urinary and blood reproductive hormone concentrations (E1-3G). Participants/materials, setting, methods Spearman and Pearson correlation coefficients (CC) and 95% confidence intervals were calculated for E1-3G and E2, and E1-3G/E2 and follicle development (total volume of ovarian follicles by ultrasound). To assess patient acceptability patient-reported outcomes (PRO) were assessed using the State Trait Anxiety Inventory (STAI) questionnaire. Further, patient-reported experiences (PRE) were evaluated through the System Usability Scale (SUS) questionnaire, along with assessing their likelihood to recommend Kinder to their friends or family undergoing IVF treatment. Main results and the role of chance Upon analysing 100 concurrent urine and blood samples from 25 patients, a robust correlation was found between E1-3G and E2 levels (Spearman CC: 0.818, 95% CI: 0.680-0.904; Pearson CC: 0.847, 95% CI: 0.712-0.939). Additionally, we observed a moderate correlation between E1-3G levels and ovarian volume (Spearman CC: 0.701, 95% CI: 0.556-0.805; Pearson CC: 0.670, 95% CI: 0.519-0.781, n = 72), resembled the correlation between E2 levels and ovarian volume (Spearman CC: 0.874, 95% CI: 0.804-0.920; Pearson CC: 0.849, 95% CI: 0.770-0.902, n = 75). PRE: 22 out of 23 patients who completed the survey confirmed that they would definitely, or probably, recommend Kinder. The mean SUS score (91.4) signified an excellent user experience. PRO: Anxiety levels were not affected by use of Kinder (STAI score below threshold of 42.38) in 18/20 patients who completed three STAI surveys (first one before and second one after the initial Kinder test; third one after the third/fourth Kinder test). Although 2/20 patients reported a STAI score of 49 in the final survey, suggesting increased anxiety, both affirmed in the PRE survey that they “will definitely recommend Kinder”, suggesting that the elevated STAI score can be attributed to the overall IVF treatment rather than the use of Kinder itself. Limitations, reasons for caution The sample size is relatively small, necessitating further multi-centre studies with a larger sample size to validate the findings. Wider implications of the findings The results suggest an acceptable interchangeability between at-home urine E1-3G and at-clinic blood E2 monitoring hormone levels during OS treatment. A digital health approach, combining tele-counselling, remote at-home hormone monitoring, and follicle tracking, could reduce patient burden and save many patients the inconvenience of travelling large distances for cycle monitoring. Trial registration number ACTRN12622000488707

Data-Driven Web-Based Service Travel Resume Information System

The Energy and Mineral Resources Department (ESDM) of South Sumatra Province has significantly enhanced its operational efficiency by implementinga Web-Based Official Travel Resume Information System. This system was developed to address the inefficiencies and risks associated with the previous manual method of managing official travel data. The new system leverages PHP, MySQL, Javascript, JQuery, and Bootstrap to provide a robust, secure, and user-friendly platform for handling travel reports, assignment orders, and official travel documentation. The digital transformation has streamlined data entry, improved data integrity, and facilitated real-time information access,enabling better decision-making and strategic planning. A System Usability Scale (SUS) test was conducted with 120 participants to evaluate the system's usability. The results showed a mean SUS score of 78.5, indicating good usability, with a standard deviation of 8.3. High scores were recorded for the system's ease of use, integration of functions, and user confidence, while lower scores highlighted initial complexity and a need for additional user training. The successful implementation and positive usability results underscore the system's effectiveness in enhancing the ESDM Department's operational capabilities. Recommendations for further improvements include enhancing mobile accessibility, providing ongoing user training, integrating advanced reporting features, and refining the user interface to ensure continued user satisfaction and system efficiency.

Development and usability testing of an online support tool to identify models and frameworks to inform implementation

BackgroundTheories, models and frameworks (TMFs) are useful when implementing, evaluating and sustaining healthcare evidence-based interventions. Yet it can be challenging to identify an appropriate TMF for an implementation project. We developed and tested the usability of an online tool to help individuals who are doing or supporting implementation practice activities to identify appropriate models and/or frameworks to inform their work.MethodsWe used methods guided by models and evidence on implementation science and user-centered design. Phases of tool development included applying findings from a scoping review of TMFs and interviews with 24 researchers/implementers on barriers and facilitators to identifying and selecting TMFs. Based on interview findings, we categorized the TMFs by aim, stage of implementation, and target level of change to inform the tool’s algorithm. We then conducted interviews with 10 end-users to test the usability of the prototype tool and administered the System Usability Scale (SUS). Usability issues were addressed and incorporated into the tool.ResultsWe developed Find TMF, an online tool consisting of 3–4 questions about the user’s implementation project. The tool’s algorithm matches key characteristics of the user’s project (aim, stage, target change level) with characteristics of different TMFs and presents a list of candidate models/frameworks. Ten individuals from Canada or Australia participated in usability testing (mean SUS score 84.5, standard deviation 11.4). Overall, participants found the tool to be simple, easy to use and visually appealing with a useful output of candidate models/frameworks to consider for an implementation project. Users wanted additional instruction and guidance on what to expect from the tool and how to use the information in the output table. Tool improvements included incorporating an overview figure outlining the tool steps and output, displaying the tool questions on a single page, and clarifying the available functions of the results page, including adding direct links to the glossary and to complementary tools.ConclusionsFind TMF is an easy-to-use online tool that may benefit individuals who support implementation practice activities by making the vast number of models and frameworks more accessible, while also supporting a consistent approach to identifying and selecting relevant TMFs.

The feasibility and usability of mixed reality teaching in a hospital setting based on self-reported perceptions of medical students

BackgroundClinical teaching during encounters with real patients lies at the heart of medical education. Mixed reality (MR) using a Microsoft HoloLens 2 (HL2) offers the potential to address several challenges: including enabling remote learning; decreasing infection control risks; facilitating greater access to medical specialties; and enhancing learning by vertical integration of basic principles to clinical application. We aimed to assess the feasibility and usability of MR using the HL2 for teaching in a busy, tertiary referral university hospital.MethodsThis prospective observational study examined the use of the HL2 to facilitate a live two-way broadcast of a clinician-patient encounter, to remotely situated third and fourth year medical students. System Usability Scale (SUS) Scores were elicited from participating medical students, clinician, and technician. Feedback was also elicited from participating patients. A modified Evaluation of Technology-Enhanced Learning Materials: Learner Perceptions Questionnaire (mETELM) was completed by medical students and patients.ResultsThis was a mixed methods prospective, observational study, undertaken in the Day of Surgery Assessment Unit. Forty-seven medical students participated. The mean SUS score for medical students was 71.4 (SD 15.4), clinician (SUS = 75) and technician (SUS = 70) indicating good usability. The mETELM Questionnaire using a 7-point Likert Scale demonstrated MR was perceived to be more beneficial than a PowerPoint presentation (Median = 7, Range 6–7). Opinion amongst the student cohort was divided as to whether the MR tutorial was as beneficial for learning as a live patient encounter would have been (Median = 5, Range 3–6). Students were positive about the prospect of incorporating of MR in future tutorials (Median = 7, Range 5–7). The patients’ mETELM results indicate the HL2 did not affect communication with the clinician (Median = 7, Range 7–7). The MR tutorial was preferred to a format based on small group teaching at the bedside (Median = 6, Range 4–7).ConclusionsOur study findings indicate that MR teaching using the HL2 demonstrates good usability characteristics for providing education to medical students at least in a clinical setting and under conditions similar to those of our study. Also, it is feasible to deliver to remotely located students, although certain practical constraints apply including Wi-Fi and audio quality.

Augmented Reality-Guided Extraction of Fully Impacted Lower Third Molars Based on Maxillofacial CBCT Scans

(1) Background: This study aimed to integrate an augmented reality (AR) image-guided surgery (IGS) system, based on preoperative cone beam computed tomography (CBCT) scans, into clinical practice. (2) Methods: In preclinical and clinical surgical setups, an AR-guided visualization system based on Microsoft’s HoloLens 2 was assessed for complex lower third molar (LTM) extractions. In this study, the system’s potential intraoperative feasibility and usability is described first. Preparation and operating times for each procedure were measured, as well as the system’s usability, using the System Usability Scale (SUS). (3) Results: A total of six LTMs (n = 6) were analyzed, two extracted from human cadaver head specimens (n = 2) and four from clinical patients (n = 4). The average preparation time was 166 ± 44 s, while the operation time averaged 21 ± 5.9 min. The overall mean SUS score was 79.1 ± 9.3. When analyzed separately, the usability score categorized the AR-guidance system as “good” in clinical patients and “best imaginable” in human cadaver head procedures. (4) Conclusions: This translational study analyzed the first successful and functionally stable application of the HoloLens technology for complex LTM extraction in clinical patients. Further research is needed to refine the technology’s integration into clinical practice to improve patient outcomes.

Effectiveness of the HEAR-Aware App for Adults Not Ready for Hearing Aids, but Open to Self-Management Support: Results of a Randomized Controlled Trial.

Recently, the HEAR-aware app was developed to support adults who are eligible for hearing aids (HAs) but not yet ready to use them. The app serves as a self-management tool, offering assistance for a range of target behaviors (TBs), such as communication strategies and emotional coping. Using ecological momentary assessment and intervention, the app prompts users to complete brief surveys regarding challenging listening situations they encounter in their daily lives (ecological momentary assessment). In response, users receive educational content in the form of "snippets" (videos, texts, web links) on the TBs, some of which are customized based on the reported acoustic environmental characteristics (ecological momentary intervention). The primary objective of this study was to assess the effectiveness of the HEAR-aware app in enhancing readiness to take action on various TBs and evaluate its impact on secondary outcomes. The secondary objective was to examine the app's usability, usefulness, and user satisfaction. A randomized controlled trial design with two arms was used. Participants with hearing loss aged 50 years and over were recruited via an HA retailer and randomly assigned to the intervention group (n = 42, mean age = 65 years [SD = 9.1]) or the control group (n = 45, mean age = 68 years [SD 8.7]). The intervention group used the app during 4 weeks. The control group received no intervention. All participants completed online questionnaires at baseline (T0), after 4 weeks (T1), and again 4 weeks later (T2). Participants' readiness to take action on five TBs was measured with The Line Composite. A list of secondary outcomes was used. Intention-to-treat analyses were performed using Linear Mixed effect Models including group (intervention/control), time (T0/T1/T2), and Group × Time Interactions. In addition, a per protocol analysis was carried out to explore whether effects depended on app usage. For the secondary aim the System Usability Scale (SUS), the Intrinsic Motivation Inventory, item 4 of the International Outcome Inventory-Alternative Intervention (IOI-AI), and a recommendation item were used (intervention group only at T1). For objective 1, there was no significant group difference for The Line Composite over the course of T0, T1, and T2. However, a significant (p = 0.033) Group × Time Interaction was found for The Line Emotional coping, with higher increase in readiness to take action on emotional coping in the intervention group than in the control group. The intention-to-treat analyses revealed no other significant group differences, but the per protocol analyses showed that participants in the intervention group were significantly more ready to take up Assistive Listening Devices (The Line Assistive Listening Devices) and less ready to take up HAs (Staging Algorithm HAs) than the control group (p = 0.049). Results for objective 2 showed that on average, participants rated the app as moderately useful (mean Intrinsic Motivation Inventory score 5 out of 7) and its usability as "marginal" (mean SUS score 68 out of 100) with about half of the participants rating the app as "good" (SUS score >70) and a minority rating is as "unacceptable" (SUS score ≤50). This study underscores the potential of self-management support tools like the HEAR-aware app in the rehabilitation of adults with hearing loss who are not yet ready for HAs. The range in usability scores suggest that it may not be a suitable intervention for everyone.

#2978 Formative usability test of a novel device for peritoneal dialysis (WEAKID)

Abstract Background and Aims Chronic kidney disease is a growing problem worldwide with a growing number of patients requiring kidney replacement therapy. In recent years we have developed a novel device for peritoneal dialysis (PD), the WEarable Artificial KIDney (WEAKID); composed of a wearable device connected to a dialysate reservoir. WEAKID uses both sorbent technology and continuous flow PD to improve dialysis efficacy. Formative usability testing aids to the identification of design issues that may interfere with a device's safe and effective use and thus may influence decisions made in the design process. Here, we present the results of an international, multicentre, formative usability study of WEAKID among nurses and patients. Method This study used a mixed-methods design and was conducted in hospitals in Italy (Modena University Hospital), Spain (La Paz University Hospital, Madrid), and the Netherlands (University Medical Center Utrecht) between November 2021 and December 2022. An individual, formative usability testing session was conducted with each participant (nurse or patient) under the conditions of simulated use. Participants were asked to perform basic operational procedures (i.e. turning the device on/off, input of dialysis settings, handling an alarm) with the device according to a usability task list. Their performance was observed and use errors (i.e., something that a participant does or fails to do that results in an unexpected or unintended outcome) were noted. Furthermore, participants were asked for feedback, both verbally and by means of completing the system usability scale (SUS), reflecting perceived usability on a scale from 0 (worst) to 100 (best). Results We included 14 participants (n = 6 nurses, n = 8 patients) in three participating centres. All nurses were female and had a median experience with PD of 20.5 years (range 0.5-32). Patients were generally highly educated (n = 5, 63%). In total, 22 use errors occurred, 10 in nurses and 12 in patients. These use errors included factors not related to the device, e.g. not taking appropriate hygiene measures. In particular, use errors occurred during the priming/connection of catheter and handling the alarm. Participants’ feedback mainly concerned the small screen (and thus font) size, user interface (e.g. lack of colour-coded soft buttons), and (short) length of the patient line. The mean SUS score was 70 (range 57.5–85) for nurses, and 71.9 (range 55–87.5) for patients. Conclusion Despite the slightly above-average perceived usability of the device's current design stage, participants made a lot of suggestions for improvements, and usability must be improved to make the device ready for market introduction.

Clinical Utility and Usability of the Digital Box and Block Test: Mixed Methods Study.

The Box and Block Test (BBT) is a clinical tool used to measure hand dexterity, which is often used for tracking disease progression or the effectiveness of therapy, particularly benefiting older adults and those with neurological conditions. Digitizing the measurement of hand function may enhance the quality of data collection. We have developed and validated a prototype that digitizes this test, known as the digital BBT (dBBT), which automatically measures time and determines and displays the test result. This study aimed to investigate the clinical utility and usability of the newly developed dBBT and to collect suggestions for future improvements. A total of 4 occupational therapists participated in our study. To evaluate the clinical utility, we compared the dBBT to the BBT across dimensions such as acceptance, portability, energy and effort, time, and costs. We observed therapists using the dBBT as a dexterity measurement tool and conducted a quantitative usability questionnaire using the System Usability Scale (SUS), along with a focus group. Evaluative, structured, and qualitative content analysis was used for the qualitative data, whereas quantitative analysis was applied to questionnaire data. The qualitative and quantitative data were merged and analyzed using a convergent mixed methods approach. Overall, the results of the evaluative content analysis suggested that the dBBT had a better clinical utility than the original BBT, with ratings of all collected participant statements for the dBBT being 45% (45/99) equal to, 48% (48/99) better than, and 6% (6/99) lesser than the BBT. Particularly in the subcategories "acceptance," "time required for evaluation," and "purchase costs," the dBBT was rated as being better than the original BBT. The dBBT achieved a mean SUS score of 83 (95% CI 76-96). Additionally, several suggested changes to the system were identified. The study demonstrated an overall positive evaluation of the clinical utility and usability of the dBBT. Valuable insights were gathered for future system iterations. These pioneering results highlight the potential of digitizing hand dexterity assessments.

Pilot study evaluating the usability of MonŒil, a ChatGPT-based education tool in ophthalmology

ImportanceThere is an increasing use of artificial intelligence (AI) in ophthalmology to respond to the needs of patients to access reliable, easy-to-understand medical information. ObjectiveTo assess patient satisfaction with the usability of MonŒil, an AI-based platform designed for patient education in ophthalmology. DesignThis was a pilot cross-sectional study of the usability of MonŒil by patients followed for advanced age-related macular degeneration (AMD). MonŒil is based on ChatGPT-4 technology with specific ophthalmology-focused customizations and accessibility enhancements, and is freely available at monoeil.help. Patients were given 20 min of unsupervised interaction with no prior training or guidance, after which their feedback was collected. SettingThe study was performed in the ophthalmology department at the Creteil University Hospital. ParticipantsParticipants included 54 patients diagnosed with advanced AMD defined presenting for follow-up. Patients had to be older than 50 years of age, capable of giving informed consent, and able to understand and interact with MonŒil. Exclusion criteria were severe visual and cognitive impairment that prevented interaction with MonŒil. Main Outcome(s) and Measure(s)The primary outcome was the usability of MonŒil as measured by the System Usability Scale (SUS) questionnaire. ResultsOf the 54 participants, 34 were female (62.96 %). The mean age of the cohort was 77.76±8.14 years (range 58 to 97 years). The mean SUS score was 90.23±12.04 with a median of 92.50 (range 42.50 to 100.00), indicating excellent usability. There was a positive relationship between visual acuity and SUS score (regression coefficient 0.30 (95 % CI 0.08 to 0.51), r2=0.19, p = 0.0077). Conclusions and RelevanceMonŒil demonstrated excellent usability and satisfaction in a sample population of elderly patients with advanced AMD. These results suggest that AI-based tools like MonŒil can enhance patient education with minimal oversight in a complex field like ophthalmology, supporting its use as an adjunct to the physician-patient discussion. Further studies may be necessary to establish the applicability of MonŒil to a broader user base, and to assess its usefulness and clinical impact on patient outcomes such as patient knowledge and vision-related quality of life.

Interactive Malaysian Childhood Healthy Lifestyle (i-MaCHeL) programme: a single-arm pilot study

BackgroundThe growing obesity epidemic in Malaysia presents a public health challenge that requires innovative intervention strategies. In an effort to address this problem, an Interactive Malaysian Childhood Healthy Lifestyle (i-MaCHeL) programme, which is a web-based initiative designed for preschool child–parent dyads offers a novel approach. Nevertheless, the success of such a web-based intervention depends on several interrelated factors. This research aims to examine the feasibility of i-MaCHeL in the Malaysian context, its usability for preschool child–parent dyads, and the acceptability of the programme among these user groups.MethodsThis was a single-arm pilot study involving 46 child–parent dyads recruited from six government preschools in Terengganu, Malaysia. The preschools were selected using a cluster random sampling technique at the preschool level. The intervention feasibility was determined based on the retention rate of participants in the pilot study. The System Usability Scale (SUS) and intervention process evaluation were used to assess the usability and acceptability of the web-based i-MaCHeL programme.ResultsThe retention data demonstrated that 42 out of 46 participants completed the 13-week intervention programme, which showed that the overall retention rate was 91.3%. A mean (SD) SUS score of 84.70 (13.82) was obtained from parents, indicating that the web-based i-MaCHeL had an acceptable usability level. The mean scores of the process evaluation items ranged from 4.52 (0.63) to 4.83 (0.38), demonstrating that the web-based i-MaCHeL was highly accepted by the parents. The acceptability data also indicated that at least 92.9% (39/42) of the parents agreed/strongly agreed that the web content, programme duration, intervention dose, WhatsApp group, and delivery mode were appropriate.ConclusionsAccording to these findings, the i-MaCHeL intervention using a web-based approach was feasible, usable, and acceptable as part of a weight-related behaviour change intervention for preschool child–parent dyads. This pilot study demonstrated that the web-based i-MaCHeL was feasible and promising for delivering weight-related behavioural intervention to child–parent dyads.Trial registrationClinicalTrials.gov, NCT04711525. Registered on January 15, 2021.