Development of the ERATbi App, a Clinical Decision Support System for Early Recovery After Traumatic Brain Injury in the ICU: Usability Study.

BackgroundPeople with disabilities experience numerous barriers to being physically active, such as transportation issues, a lack of trained exercise professionals who understand disabilities, and facility access. The use of a virtual exercise platform (VEP) may provide an alternative and limit certain barriers.ObjectiveThe aim of this mixed method study was to evaluate user interaction (effectiveness, efficiency, and satisfaction), the strengths and weaknesses of the user interface, and the user experience with a VEP.MethodsParticipants were recruited from a community fitness facility that offers programs for people with disabilities. Inclusion criteria were being older than 18 years, fluent in English, and availability of internet access. Features of the VEP included articles, prerecorded videos, live Zoom classes, web-based class registration, weekly progress tracking, incentives, and surveys. A one-on-one Zoom session was scheduled with each participant, during which they completed certain tasks: (1) create an account or login, (2) register for class, (3) join class, (4) add to calendar, and (5) complete surveys. As participants completed tasks, quantitative observations (time on task, task success, rate of task completion, and number of errors by users, which determined task difficulty), qualitative observations were made and interviews were conducted at the end of the session. The “concurrent think-aloud” method was encouraged by the moderator to gauge participants’ thoughts as they worked through testing. Participants also completed the System Usability Scale (SUS) and Questionnaire for User Interface Satisfaction (QUIS).ResultsA total of 5 people with disabilities (3 male, 2 female), aged 36-78 (mean 54) years, with education levels from high school to PhD, were recruited. Devices used for testing included a laptop (n=3), a Chromebook (n=1), and a desktop (n=1). All participants completed tasks #1 and #2 without errors but could not complete task #4. One participant completed task #5 with difficulty and another completed task #3 with difficulty. The average time to complete each task was: (1) 82 seconds (55-110), (2) 11 seconds (4-21), (3) 9 seconds (5-27), and (4) 921.5 seconds (840-958). The mean SUS score was 86.5/100, and the mean user QUIS score was 8.08 out of 10. Qualitative observations indicated that the system was simple, user-friendly, and accessible.ConclusionsPeople with disabilities reported high usability and user satisfaction with the web-based exercise platform, and the system appears to be an efficient and effective physical activity option.

BackgroundComputerized clinical decision support systems (CDSSs) are increasingly adopted in health care to optimize resources and streamline patient flow. However, they often lack scientific validation against standard medical care.ObjectiveThe purpose of this study was to assess the performance, safety, and usability of a CDSS in a university hospital emergency department setting in Kuopio, Finland.MethodsPatients entering the emergency department were asked to voluntarily participate in this study. Patients aged 17 years or younger, patients with cognitive impairments, and patients who entered the unit in an ambulance or with the need for immediate care were excluded. Patients completed the CDSS web-based form and usability questionnaire when waiting for the triage nurse’s evaluation. The CDSS data were anonymized and did not affect the patients’ usual evaluation or treatment. Retrospectively, 2 medical doctors evaluated the urgency of each patient’s condition by using the triage nurse’s information, and urgent and nonurgent groups were created. The International Statistical Classification of Diseases, Tenth Revision diagnoses were collected from the electronic health records. Usability was assessed by using a positive version of the System Usability Scale questionnaire.ResultsIn total, our analyses included 248 patients. Regarding urgency, the mean sensitivities were 85% and 19%, respectively, for urgent and nonurgent cases when assessing the performance of CDSS evaluations in comparison to that of physicians. The mean sensitivities were 85% and 35%, respectively, when comparing the evaluations between the two physicians. Our CDSS did not miss any cases that were evaluated to be emergencies by physicians; thus, all emergency cases evaluated by physicians were evaluated as either urgent cases or emergency cases by the CDSS. In differential diagnosis, the CDSS had an exact match accuracy of 45.5% (97/213). The usability was good, with a mean System Usability Scale score of 78.2 (SD 16.8).ConclusionsIn a university hospital emergency department setting with a large real-world population, our CDSS was found to be equally as sensitive in urgent patient cases as physicians and was found to have an acceptable differential diagnosis accuracy, with good usability. These results suggest that this CDSS can be safely assessed further in a real-world setting. A CDSS could accelerate triage by providing patient-provided data in advance of patients’ initial consultations and categorize patient cases as urgent and nonurgent cases upon patients' arrival to the emergency department.

The Pharmaceutical Services Program (PSP) has not yet evaluated the usability of the web application system. The System Usability Scale (SUS) and Net Promoter Score (NPS) assessment tools give you a quick overview of your system's usability and help you improve it further. The goal of this study was to assess the usability of the PRiSMA and My.Pharma-C systems using SUS and NPS. From April 2019 to July 2020, a survey was conducted using a validated self-administered questionnaire that was prompted after each successful log-out from the PRiSMA and My.Pharma-C systems. The questionnaire was divided into three sections: demographic, device type, and SUS and NPS questionnaire. The mean SUS and NPS scores were computed. The survey was completed by 3,959 PRiSMA users and 811 My.Pharma-C users. The SUS score was interpreted as a grade, adjective, and range of acceptability. The PRiSMA mean SUS score was 70.2 (SD 15.44), indicating a 'Grade C' and 'Good' system. The mean SUS score for My.Pharma-C was 58.2 (SD 15.13), indicating a 'Grade D' and 'OK' system. Users accepted both systems on the basis of their usability. Calculated NPS value for PRiSMA was p=42%, n=49% and d=9%, NPS value=+33, whereas My.Pharma-C p=18%, n=46% and d=36%, NPS value=-18. A positive NPS value for PRiSMA represents the user's readiness to recommend the system to others compared to My.Pharma-C. The findings also established that SUS alone is insufficient and needs to be complemented by another method, such as qualitative instruments. It could help to drill down in depth to understand how to address potentially problematic areas for web application systems and assist in the improvement phases. However, both of the tools are applicable to be used to evaluate the usability of the web application systems that managed by the Pharmaceutical Services Program.

BackgroundThe System Usability Scale (SUS) is a widely used scale that has been used to quantify the usability of many software and hardware products. However, the SUS was not specifically designed to evaluate mobile apps, or in particular digital health apps (DHAs).ObjectiveThe aim of this study was to examine whether the widely used SUS distribution for benchmarking (mean 68, SD 12.5) can be used to reliably assess the usability of DHAs.MethodsA search of the literature was performed using the ACM Digital Library, IEEE Xplore, CORE, PubMed, and Google Scholar databases to identify SUS scores related to the usability of DHAs for meta-analysis. This study included papers that published the SUS scores of the evaluated DHAs from 2011 to 2021 to get a 10-year representation. In total, 117 SUS scores for 114 DHAs were identified. R Studio and the R programming language were used to model the DHA SUS distribution, with a 1-sample, 2-tailed t test used to compare this distribution with the standard SUS distribution.ResultsThe mean SUS score when all the collected apps were included was 76.64 (SD 15.12); however, this distribution exhibited asymmetrical skewness (–0.52) and was not normally distributed according to Shapiro-Wilk test (P=.002). The mean SUS score for “physical activity” apps was 83.28 (SD 12.39) and drove the skewness. Hence, the mean SUS score for all collected apps excluding “physical activity” apps was 68.05 (SD 14.05). A 1-sample, 2-tailed t test indicated that this health app SUS distribution was not statistically significantly different from the standard SUS distribution (P=.98).ConclusionsThis study concludes that the SUS and the widely accepted benchmark of a mean SUS score of 68 (SD 12.5) are suitable for evaluating the usability of DHAs. We speculate as to why physical activity apps received higher SUS scores than expected. A template for reporting mean SUS scores to facilitate meta-analysis is proposed, together with future work that could be done to further examine the SUS benchmark scores for DHAs.

BackgroundA healthy diet throughout the life course improves health and reduces the risk of disease. There is a need for new knowledge of the relation between diet and health, but existing methods to collect information on food and nutrient intake have their limitations. Evaluations of new tools to assess dietary intake are needed, especially in old people, where the introduction of new technology might impose challenges.ObjectiveWe aimed to examine the usability of a new web-based dietary assessment tool in older adult women and men.MethodsA total of 60 women and men (participation 83%, 57% women) aged 60-74 years recruited by convenience and snowball sampling completed a 24-hour web-based dietary recall using the newly developed Norwegian version of Measure Your Food On One Day (myfood24). Total energy and nutrient intakes were calculated in myfood24, primarily on the basis of the Norwegian Food Composition Table. No guidance or support was provided to complete the recall. Usability was assessed using the system usability scale (SUS), where an SUS score of ≥68 was considered satisfactory. We examined the responses to single SUS items and the mean (SD) SUS score in groups stratified by sex, age, educational level, and device used to complete the recall (smartphone, tablet device, or computer).ResultsThe mean total energy intake was 5815 (SD 3093) kJ. A total of 14% of participants had an energy intake of <2100 kJ (ie, 500 kilocalories) and none had an intake of >16,800 kJ (ie, 4000 kilocalories). Mean energy proportions from carbohydrates, fat, protein, alcohol, and fiber was within the national recommendations. The mean SUS score was 55.5 (SD 18.6), and 27% of participants had SUS scores above the satisfactory product cut-off. Higher SUS scores were associated with younger age and lower education, but not with the type of device used.ConclusionsWe found the overall usability of a new web-based dietary assessment tool to be less than satisfactory in accordance with standard usability criteria in a sample of 60-74–year-old Norwegians. The observed total energy intakes suggest that several of the participants underreported their intake during the completion of the dietary recall. Implementing web-based dietary assessment tools in older adults is feasible, but guidance and support might be needed to ensure valid completion.

Microinteraction Ecological Momentary Assessment (miEMA) addresses the challenges of traditional self-report questionnaires by collecting data in real time. The purpose of this study was to examine the feasibility and usability of employing miEMA using a smartwatch in military service members undergoing traumatic brain injury (TBI) rehabilitation. Twenty-eight United States active duty service members with a TBI history were recruited as patients from a military outpatient TBI rehabilitation center, enrolled in either a 2-week or 3-week study arm, and administered miEMA surveys via a custom smartwatch app. The 3-week arm participants were also concurrently receiving cognitive rehabilitation. Select constructs evaluated with miEMA included mood, fatigue, pain, headache, self-efficacy, and cognitive strategy use. Outcome measures of adherence were completion (percentage of questions answered out of questions delivered) and compliance (percentage of questions answered out of questions scheduled). The Mobile Health Application Usability Questionnaire (MAUQ) and System Usability Scale (SUS) assessed participants' perceptions of smartwatch and app usability. Completion and compliance rates were 80.1% and 77.4%, respectively. Mean participant completion and compliance were 81.1% ± 12.0% and 78.1% ± 13.0%, respectively. Mean participant completion increased to 87.7% ± 8.8% when using an embedded question retry mechanism. Mean participant survey set completion was 69.8% ± 18.3% during the early morning but remained steady during the late morning/early afternoon (85.7% ± 12.8%), afternoon (86.2% ± 12.6%), and late afternoon/evening (85.0% ± 14.7%). The mean overall item score for the MAUQ was 6.3 ± 1.1 out of 7. The mean SUS score was 89.0 ± 7.2 out of 100 and mean SUS percentile ranking was 96.4% ± 8.4%. Overall adherence was similar to previous studies in civilian populations. Participants rated the miEMA app and smartwatch as having high usability. These findings suggest that miEMA using a smartwatch for tracking symptoms and treatment strategy use is feasible in military service members with a TBI history, including those undergoing rehabilitation for cognitive difficulties.

National survey of the awareness and implementation status of early pulmonary rehabilitation for patients with critical illness in departments of pulmonary and critical care medicine in 2019.

The Neuroendocrine Effects of Traumatic Brain Injury

Neuroendocrine dysfunction after traumatic brain injury (TBI) is under-diagnosed, under-treated, and may adversely affect the rate of recovery. Single or multiple pituitary-target hormone disruption occurs in up to two-thirds of persons with TBI, most commonly affecting the gonadal and growth hormone axes. The time course of decline in and recovery of pituitary function in relation to cognitive dysfunction and rehabilitation progress are not well described. This article reviews the clinical spectrum of neuroendocrine deficits after TBI and their underlying mechanisms. Future studies of the effects of hormonal replacement on recovery are recommended.

BackgroundTransparency is increasingly called for in health care, especially, when it comes to patients’ access to their electronic health records. In Sweden, the e-service Journalen is a national patient accessible electronic health record (PAEHR), accessible online via the national patient portal. User characteristics and perceived benefits of using a PAEHR influence behavioral intention for use and adoption, but poor usability that increases the effort expectancy can have a negative impact. It is, therefore, of interest to explore how users of the PAEHR Journalen perceive its usability and usefulness.ObjectiveThe aim of this study was to explore how the users of the Swedish PAEHR experience the usability of the system and to identify differences in these experiences based on the level of transparency of the region.MethodsA survey study was conducted to elicit opinions and experiences of patients using Journalen. The data were collected from June to October 2016. The questionnaire included questions regarding the usability of the system from the System Usability Scale (SUS). The SUS analysis was the focus of this paper. Analysis was performed on different levels: nationally looking at the whole data set and breaking it down by focusing on 2 different regions to explore differences in experienced usability based on the level of transparency.ResultsDuring the survey period, 423,141 users logged into Journalen, of which 2587 unique users completed the survey (response rate 0.61%). The total mean score for all respondents to the SUS items was 79.81 (SD 14.25), which corresponds to a system with good usability. To further explore whether the level of transparency in a region would affect the user’s experience of the usability of the system, we analyzed the 2 regions with the most respondents: Region Uppsala (the first to launch, with a high level of transparency), and Region Skåne (an early implementer, with a low level of transparency at the time of the survey). Of the participants who responded to at least 1 SUS statement, 520 stated that they had received care in Region Skåne, whereas 331 participants had received care in Region Uppsala. Uppsala’s mean SUS score was 80.71 (SD 13.41), compared with Skåne’s mean of 79.37 (SD 13.78).ConclusionsThe Swedish national PAEHR Journalen has a reasonably good usability (mean SUS score 79.81, SD 14.25); however, further research into more specific usability areas are needed to ensure usefulness and ease of use in the future. A somewhat higher SUS score for the region with high transparency compared with the region with low transparency could indicate a relationship between the perceived usability of a PAEHR and the level of transparency offered, but further research on the relationship between transparency and usability is required.

Mobile health applications offer valuable support for clinical decision-making. However, usability and perceived usefulness influence adoption. The aim of this study was to evaluate the usability and perceived usefulness of the DentalHelp web app, a clinical decision support system (CDSS) for managing periapical radiolucencies associated with root-filled teeth. A cross-sectional study was conducted with 51 third-year dental students. Participants completed predefined clinical tasks using DentalHelp and assessed its usability and perceived usefulness through the System Usability Scale (SUS) and an adapted Technology Acceptance Model questionnaire. Statistical analyses examined correlations between demographic factors and app evaluation scores. The mean SUS score was 96/100, indicating excellent usability. Perceived usefulness was rated 4.65/5, with high agreement on its effectiveness in clinical decision-making. DentalHelp demonstrated high usability and usefulness, supporting its role as a clinical and educational tool. DentalHelp is a highly usable and useful CDSS for managing periapical radiolucencies, with potential applications in both education and practice. Further studies should explore its impact on clinical outcomes and its long-term adoption.

Early intervention can significantly improve long-term outcomes for children with autism. Unfortunately, many children do not receive early intervention services due to a critical shortage of trained professionals in this area. To bridge this gap, we evaluated a Web-based parent training tutorial (Enhancing Interactions), based on evidence-based practices and utilizing the Web-based platform to maximize learning. Twenty-three parents with a child between 18 months and 6 years with an autism spectrum disorder participated. Pre- and posttest scores of parents' knowledge were used to evaluate tutorial effectiveness. The system usability scale (SUS) evaluated technical user-friendliness and the user satisfaction questionnaire (USQ), gauged satisfaction with content. The mean number of correct items on the posttest significantly increased, from 12.6 to 20.4, p<0.001. The mean SUS score was 85 (standard deviation=17), corresponding to a score of "excellent." All participants found the tutorial user friendly, well integrated, and 96% (all but one participant) thought it was easy to use, felt confident using the technical features, and would use a tutorial like this again. On the USQ, all participants found that the tutorial was well organized, clearly presented, and easy to understand; that it increased their knowledge about communicating with their child; and that they felt capable of applying these techniques with their child. The tutorial appears effective in increasing parents' knowledge with high user satisfaction.

BackgroundAdolescents with asthma are vulnerable to poor asthma outcomes due to inadequate self-management skills and nonadherence to medications. Mobile health (mHealth) apps have shown promise in improving asthma control, medication adherence, and self-efficacy. However, existing mHealth asthma apps lack personalization and real-time feedback and are not tailored for at-risk adolescents.ObjectiveThis study aimed to design, develop, and test a smartphone-based mHealth Asthma Action Plan for adolescents, called Pulmonary Education and Knowledge Mobile Asthma Action Plan (PEAK-mAAP), in preparation for a large-scale randomized controlled trial.MethodsWe employed user-centered design principles to develop our app, leveraging our previous work and following guidelines from the National Heart, Lung, and Blood Institute. The app consists of a patient-facing mobile app and a provider-facing portal. A convenience sample of 13 adolescents (aged 12‐20 years) was recruited from the Arkansas Children’s Research Institute database or direct health care provider referrals. Participants underwent a task-based usability assessment followed by the System Usability Scale assessment to measure user satisfaction, interface effectiveness, and overall system usability.ResultsPEAK-mAAP integrates 7 core modules supporting personalized asthma self-management, symptom monitoring, medication tracking, and real-time feedback. The mean System Usability Scale score was 83/100 (SD 5.54), indicating high user satisfaction and system usability. Notably, older adolescents (>17 years) reported higher usability scores (87.5) than younger users (77.5), suggesting potential age-related differences in app navigation and engagement.ConclusionsThe results demonstrate that PEAK-mAAP is a feasible and user-friendly mHealth intervention for adolescent asthma self-management. While the high usability score reflects a positive user experience, some participants encountered initial usability challenges, highlighting the need for minor refinements and user training materials. The integration of personalized self-management tools and real-time feedback distinguishes PEAK-mAAP from existing asthma apps, addressing key barriers to adherence and engagement. Moving forward, an ongoing randomized controlled trial will assess its clinical effectiveness, long-term engagement, and impact on asthma outcomes, providing further insights into its potential as a scalable solution for adolescent asthma care.

Artificial intelligence in combination with modern technologies including medical screening devices has the potential to deliver better management services to deal with chronic diseases with higher accuracy, efficiency, and satisfaction. With the recent evolution in digitized data acquisition, computer vision and machine learning, AI solutions are spreading into areas which were previously examined by well-trained clinicians. Early diagnosis of diabetic retinopathy (DR) and foot ulcers (DFU) occurrence through image analysis is in high demand as many individuals are left without any supervision due to the limited resources such as trained clinicians or suitable equipment especially, in rural areas. Furthermore, the existing system will become even more insufficient as the number of people with diabetes increases. In this research paper, we propose a prototype that involves an autonomous system called an Intelligent Diabetic Assistant (IDA), which decides the diagnosis and the treatment prioritization depending upon the observations appeared in the screen. The IDA consists of knowledge-based modules for severity level-based classification, clinical decision support and near real-time foot ulcer detection and boundary screening. We use the System Usability Scale (SUS) in terms of performance, learnability, and satisfaction to measure the usability of the IDA. The mean SUS score was 88.5, demonstrating good but not exceptional system usability. We perform our experiments with clinicians who have been involved in diabetic care.

Funding Acknowledgements Type of funding sources: Private grant(s) and/or Sponsorship. Main funding source(s): Novartis Background/significance The rapid development of information technology is driving the evolution of choices of modes of delivery of healthcare services. The internet is an extremely powerful tool for accessing information, and communication. However, involvement of patients in the assessment of such an intervention is crucial, and can be achieved through a person-centred approach, utilising a co-design process with the aim to increase intervention usability. Purpose To assess the usability of the Country Access to Cardiac Health (CATCH) Web, an interactive web application for cardiac rehabilitation (CR), co-designed with rural and remote Australians, following the co-design development phase. Methods The User Experience (UX) Design framework was used to co-design the web portal with CR patients through two rounds of six co-design workshops pre (n=39), and post (n=35). Participants and family members living in rural and remote areas of Australia were invited to participate based on their eligibility for CR as outlined by the National Heart Foundation of Australia. As part of the UX Design Framework the System Usability Scale (SUS) was used to measure objective feedback from participants on the website design. The SUS assesses the three core components of a products usability: effectiveness, efficiency and satisfaction using a 10 question Likert scale. Mean scores were compared between the two rounds of workshops using an independent T-Test. The mean SUS score is categorised into poor to moderate (&amp;lt;68), good (68-80.3) and excellent (&amp;gt;=80.3) and categories were compared between the two rounds of workshops through Chi-Square tests. Results The 35 participants in the usability workshops had a mean age of 68.6 (SD 11.2) years and 16 (47%) were female (Table 1). The majority had experienced a myocardial infarction (15, 42.8%) and had hyperlipidemia (17, 50%), heart failure (15, 44%), hypertension (14, 41%), and Type II diabetes (7, 21%). Of these participants 20 (59%) used a smart phone, 18 (58%) had access to the National Broadband Network and 20 (59%) engaged with Facebook. From the first to the second round of workshops, there was an improvement in the mean SUS scores from 66.7 (SD16.8) to 73.6 (SD 21.0); p=0.26. The proportion of participants rating it as good or excellent increased from 48.7 to 65.8%; p=0.11 (Figure1). Conclusion The usability testing of the interactive CATCH web application showed an improvement in the SUS rating from poor to moderate in the co-design development phase to good or excellent by most of the participants in the usability testing phase. Usability is closely related to engagement with a digital health intervention. Upcoming evaluation of this intervention will report on clinical outcomes.