Health Information Technology Usability Evaluation Research Articles

Wearables for the Bladder: Stakeholder Perspectives on Moving Multiple Sclerosis Bladder Dysfunction Interventions Into the 21st Century.

Bladder dysfunction (BD) is common in people with multiple sclerosis (MS) and can reduce participation in daily life. Detecting BD early allows for effective prevention-focused treatments such as pelvic floor physical therapy. Pairing neurotechnology with patient-reported outcomes to remotely measure BD could significantly improve monitoring and treatment of BD. Therefore, we describe the process and findings of stakeholder engagement from a human-centered design process to assemble a wearables for the bladder (WeB) kit. Four people with MS with varying BD severity, and 5 MS clinical/research experts had 4 virtual meetings. Commercially available bladder tools were graded for ability to evaluate, monitor, or treat BD. The Health Information Technology Usability Evaluation Scale (utility, usability, feasibility) was used for evaluation. Scoring was performed individually and as a group. Of the 11 devices, 5 obtained mean scores of greater than 6 of 10 for likability, usability, and device utility. The 2 highest scoring (9/10) devices were selected for the pilot. One device measures bladder urine levels, reporting the number/frequency of voids/leaks; the other guides pelvic floor exercises by pairing games on an app with biofeedback from intravaginal sensors. We uncovered critical differences in experts' and patients' appreciation of the tools, and the collaborative engagement led to substantial revisions of initial tool scores. This process underscores the critical role of stakeholder engagement in the selection of digital tools, especially in sensitive domains like pelvic function. Ongoing clinical validation of the selected tools will yield a validated, user-friendly WeB kit that is able to fill gaps in our ability to evaluate BD treatments in people with MS, ultimately reducing the impact of BD on quality of life.

User Interface Design and Usability Evaluation of a Mental Health Mobile Application for Healthcare Workers: A Nursing Informatics Perspective

Objectives: The critical need for accessible mental health support for healthcare workers during pandemics underscores the importance of tailored interventions. This study aimed to derive user interface (UI) design elements and evaluate the usability of a mobile mental health application specifically crafted for healthcare professionals, focusing on nursing informatics.Methods: The mental healing app (mHealing app) interfaces were developed, emphasizing UI design elements derived from user requirements and guided by principles of nursing informatics. Usability was assessed using the Health Information Technology Usability Evaluation Scale (Health-ITUES).Results: Six nurses, including one specializing in nursing informatics, participated in both UI design derivation and usability pilot test, providing valuable insights from a nursing informatics perspective. The mHealing app offers personalized content delivery and facilitates connections to counselors based on users’ mental health states, supported by a self-diagnosis tool. The UI design prioritizes user-centered features and intuitive navigation, addressing healthcare workers’ unique challenges during pandemics. In the usability pilot test, the app scored high in impact, perceived usefulness, and ease of use, according to Health-ITUES.Conclusions: The mHealing app, developed with a focus on UI design elements informed by nursing informatics principles, effectively addresses the mental health care needs of healthcare workers during pandemics, particularly COVID-19. Engaging six nurses in usability pilot test, including one specializing in nursing informatics, the app demonstrates a user-centered approach and intuitive UI, promising a meaningful impact in clinical settings.

A Nursing Clinical Care Services Platform to Support Leadership Decision Making.

Like most organizations, The Children's Hospital of Philadelphia (CHOP) generates a lot of data of varying types, from clinical to staffing and operations. However, these datasets are separated, preventing effective analytics. To improve workforce management and care delivery, CHOP's Nursing department collaborated with the Data & Analytics team to create a dashboard-based platform for analyzing workforce metrics and patient outcomes, aiding nursing leaders in decision-making. This project involved multiple phases with contributions from Nursing, Data & Analytics, CHOP's Data Trust Office, and Human Resources. Key performance indicators were identified, and a variety of data sources were aggregated to provide a comprehensive view of the nursing enterprise. The resulting platform offers automated, current, and reliable analytics on various aspects, including nursing demographics, education, survey results, staffing actuals by job group, and patient and family experience data. The platform's usability was assessed using a modified Health Information Technology Usability Evaluation Scale survey, with a 29% response rate primarily from senior directors and managers. The findings showed high usability and satisfaction, indicating the dashboard is a valuable decision-support tool. Lessons learned include the need for analytics education for nurses and mid-managers, the inclusion of critical nursing-specific metrics (and development of the data pipelines making them possible), and the integration of multidisciplinary team metrics for comprehensive nursing analytics.

Sociotechnical feasibility of natural language processing-driven tools in clinical trial eligibility prescreening for Alzheimer's disease and related dementias.

Alzheimer's disease and related dementias (ADRD) affect over 55 million globally. Current clinical trials suffer from low recruitment rates, a challenge potentially addressable via natural language processing (NLP) technologies for researchers to effectively identify eligible clinical trial participants. This study investigates the sociotechnical feasibility of NLP-driven tools for ADRD research prescreening and analyzes the tools' cognitive complexity's effect on usability to identify cognitive support strategies. A randomized experiment was conducted with 60 clinical research staff using three prescreening tools (Criteria2Query, Informatics for Integrating Biology and the Bedside [i2b2], and Leaf). Cognitive task analysis was employed to analyze the usability of each tool using the Health Information Technology Usability Evaluation Scale. Data analysis involved calculating descriptive statistics, interrater agreement via intraclass correlation coefficient, cognitive complexity, and Generalized Estimating Equations models. Leaf scored highest for usability followed by Criteria2Query and i2b2. Cognitive complexity was found to be affected by age, computer literacy, and number of criteria, but was not significantly associated with usability. Adopting NLP for ADRD prescreening demands careful task delegation, comprehensive training, precise translation of eligibility criteria, and increased research accessibility. The study highlights the relevance of these factors in enhancing NLP-driven tools' usability and efficacy in clinical research prescreening. User-modifiable NLP-driven prescreening tools were favorably received, with system type, evaluation sequence, and user's computer literacy influencing usability more than cognitive complexity. The study emphasizes NLP's potential in improving recruitment for clinical trials, endorsing a mixed-methods approach for future system evaluation and enhancements.

The Usability of a Touchpad Active Video Game Controller for Individuals With Impaired Mobility: Observational Study.

Video games are a popular sedentary activity among people with impaired mobility; however, active video game hardware typically lacks accessibility and customization options for individuals with mobility impairments. A touchpad video game system can elicit moderate physical activity in healthy adults; however, it is unclear if this system is usable by adults with impaired mobility. The purpose of this study was to assess the usability of a touchpad video game controller system adapted for adults with impaired mobility. Additional outcomes explored were enjoyment, perceived exertion, self-efficacy, participant feedback, and researcher observations of gameplay. Participants played several video game titles for 20 minutes with a touchpad video game controller as they stood or sat in a chair or their wheelchair. Usability was assessed with the System Usability Scale (SUS) and the Health Information Technology Usability Evaluation Scale (Health-ITUES) surveys after gameplay. After each video game, participants reported enjoyment using a visual analog scale (0 to 100 mm) and a rating of perceived exertion using the OMNI 0 to 10 scale. Self-efficacy was measured before and after gameplay. Participants provided feedback at the end of their session. In total, 21 adults (6 females and 15 males) with a mean age of 48.8 (SD 13.8) years with various mobility impairments participated in this study. The touchpads received mean usability scores on the SUS 80.1 (SD 18.5) and Health-ITUES 4.23 (SD 0.67). The SUS scores reported suggest the touchpad system is "usable"; however, the Health-ITUES scores were slightly below a suggested benchmark. Participants reported moderate to high enjoyment but perceived the exertion as "somewhat easy." Self-efficacy was moderate to high and did not differ pre- to postgame play. The participants regarded the touchpads as novel, fun, and entertaining. The generalizability of our results is limited due to the heterogenous sample; however, our participants identified several areas of improvement for future iteration.

"A Great Way to Start the Conversation": Evidence for the Use of an Adolescent Mental Health Chatbot Navigator for Youth at Risk of HIV and Other STIs.

Chatbot use is increasing for mobile health interventions on sensitive and stigmatized topics like mental health because of their anonymity and privacy. This anonymity provides acceptability to sexual and gendered minority youth (ages 16-24) at increased risk of HIV and other STIs with poor mental health due to higher levels of stigma, discrimination, and social isolation. This study evaluates the usability of Tabatha-YYC, a pilot chatbot navigator created to link these youth to mental health resources.Tabatha-YYC was developed using a Youth Advisory Board (n = 7). The final design underwent user testing (n = 20) through a think-aloud protocol, semi-structured interview, and a brief survey post-exposure which included the Health Information Technology Usability Evaluation Scale.The chatbot was found to be an acceptable mental health navigator by participants.This study provides important design methodology considerations and key insights into chatbot design preferences of youth at risk of STIs seeking mental health resources.

Assessing Usability and Ambulatory Clinical Staff Satisfaction with Two Electronic Health Records.

A growing body of literature has linked usability limitations within electronic health records (EHRs) to adverse outcomes which may in turn affect EHR system transitions. NewYork-Presbyterian Hospital, Columbia University College of Physicians and Surgeons (CU), and Weill Cornell Medical College (WC) are a tripartite organization with large academic medical centers that initiated a phased transition of their EHRs to one system, EpicCare. This article characterizes usability perceptions stratified by provider roles by surveying WC ambulatory clinical staff already utilizing EpicCare and CU ambulatory clinical staff utilizing iterations of Allscripts before the implementation of EpicCare campus-wide. A customized 19-question electronic survey utilizing usability constructs based on the Health Information Technology Usability Evaluation Scale was anonymously administered prior to EHR transition. Responses were recorded with self-reported demographics. A total of 1,666 CU and 1,065 WC staff with ambulatory self-identified work setting were chosen. Select demographic statistics between campus staff were generally similar with small differences in patterns of clinical and EHR experience. Results demonstrated significant differences in EHR usability perceptions among ambulatory staff based on role and EHR system. WC staff utilizing EpicCare accounted for more favorable usability metrics than CU across all constructs. Ordering providers (OPs) denoted less usability than non-OPs. The Perceived Usefulness and User Control constructs accounted for the largest differences in usability perceptions. The Cognitive Support and Situational Awareness construct was similarly low for both campuses. Prior EHR experience demonstrated limited associations. Usability perceptions can be affected by role and EHR system. OPs consistently denoted less usability overall and were more affected by EHR system than non-OPs. While there was greater perceived usability for EpicCare to perform tasks related to care coordination, documentation, and error prevention, there were persistent shortcomings regarding tab navigation and cognitive burden reduction, which have implications on provider efficiency and wellness.

Health Information Technology Usability Evaluation Scale (Health-ITUES) and User-Experience Questionnaire (UEQ) for 3D Intraoperative Cognitive Navigation (ICON3DTM) System for Urological Procedures

Backgound and objectives: In recent years, the adoption of 3D models for surgical planning and intraoperative guidance has gained a wide diffusion. The aim of this study was to evaluate the surgeons' perception and usability of ICON3DTM platform for robotic and laparoscopic urological surgical procedures. Materials and Methods: During the 10th edition of the Techno-Urology Meeting, surgeons and attendees had the opportunity to test the new ICON3DTM platform. The capability of the user to manipulate the model with hands/mouse, the software usability, the quality of the 3D model's reproduction, and the quality of its use during the surgery were evaluated with the Health Information Technology Usability Evaluation Scale (Health-ITUES) and the User-Experience Questionnaire (UEQ). Results: Fifty-three participants responded to the questionnaires. Based on the answers to the Health-ITUES questionnaire, ICON3DTM resulted to have a positive additional value in presurgical/surgical planning with 43.4% and 39.6% of responders that rated 4 (agree) and 5 (strongly agree), respectively. Regarding the UEQ questionnaire, both mouse and infrared hand-tracking system resulted to be easy to use for 99% of the responders, while the software resulted to be easy to use for 93.4% of the responders. Conclusions: In conclusion, ICON3DTM has been widely appreciated by urologists thanks to its various applications, from preoperative planning to its support for intraoperative decision-making in both robot-assisted and laparoscopic settings.

Moderated digital social therapy for young people with emerging mental health problems: A user-centered mixed-method design and usability study.

Over 25% of Dutch young people are psychologically unhealthy. Individual and societal consequences that follow from having mental health complaints at this age are substantial. Young people need care which is often unavailable. ENgage YOung people earlY (ENYOY) is a moderated digital social therapy-platform that aims to help youngsters with emerging mental health complaints. Comprehensive research is being conducted into the effects and to optimize and implement the ENYOY-platform throughout the Netherlands. The aim of this study is to explore the usability and user experience of the ENYOY-platform. A user-centered mixed-method design was chosen. 26 young people aged 16-25 with emerging mental health complaints participated. Semi-structured interviews were conducted to explore usability, user-friendliness, impact, accessibility, inclusivity, and connection (Phase 1). Phase 2 assessed usability problems using the concurrent and retrospective Think Aloud-method. User experience and perceived helpfulness were assessed using a 10-point rating scale and semi-structured interviews (Phase 3). The Health Information Technology Usability Evaluation Scale (Health-ITUES; Phase 1) and System Usability Scale (SUS; Phase 2 and 3) were administered. Qualitative data was analyzed using thematic analysis. Task completion rate and time were tracked and usability problems were categorized using the Nielsen's rating scale (Phase 2). Adequate to high usability was found (Phase 1 Health-ITUES 4.0(0.34); Phase 2 SUS 69,5(13,70); Phase 3 SUS 71,6(5,63)). Findings from Phase 1 (N = 10) indicated that users viewed ENYOY as a user-friendly, safe, accessible, and inclusive initiative which helped them reduce their mental health complaints and improve quality of life. Phase 2 (N = 10) uncovered 18 usability problems of which 5 of major severity (e.g. troubles accessing the platform). Findings from Phase 3 (N = 6) suggested that users perceived the coaching calls the most helpful [9(0.71)] followed by the therapy content [6.25(1.41)]. Users liked the social networking aspect but rated it least helpful [6(2.1)] due to inactivity. The ENYOY-platform has been found to have adequate to high usability and positive user experiences were reported. All findings will be transferred to the developmental team to improve the platform. Other evaluation methods and paring these with quantitative outcomes could provide additional insight in future research.

Triptychon: Usability evaluation and implementation of a web-based application for patients' lab and vital parameters.

A major challenge in healthcare is the interpretation of the constantly increasing amount of clinical data of interest to inpatients for diagnosis and therapy. It is vital to accurately structure and represent data from different sources to help clinicians make informed decisions. We evaluated the usability of our tool 'Triptychon' - a three-part visualisation dashboard of essential patients' medical data provided by a direct overview of their hospitalisation information, laboratory, and vital parameters over time. The study followed a cohort of 20 participants using the mixed-methods approach, including interviews and the usability questionnaires, Health Information Technology Usability Evaluation Scale (Health-ITUES), and User Experience Questionnaire (UEQ). The participant's interactions with the dashboard were also observed. A thematic analysis approach was applied to analyse qualitative data and the quantitative data's task completion time and success rates. The usability evaluation of the visualisation dashboard revealed issues relating to the terminology used in the user interface and colour coding in its left and middle panels. The Health-ITUES score was 3.72 (standard deviation (SD) = 1.0), and the UEQ score was 1.6 (SD = 0.74). The study demonstrated improvements in intuitive dashboard use and overall satisfaction with using the dashboard daily. The Triptychon dashboard is a promising new tool for medical data presentation. We identified design and layout issues of the dashboard for improving its usability in routine clinical practice. According to users' feedback, the three panels on the dashboard provided a holistic view of a patient's hospital stay.

MyGuide: Perceptions and Acceptance of an Online Concussion Self-Management Tool

<h3>Research Objectives</h3> To investigate the perceptions and acceptance of clinicians and people with concussion using an online concussion self-management tool. <h3>Design</h3> Using a mixed-methods content analysis approach, we interviewed and analyzed responses from clients and clinicians who used the online self-management tool within the past 2 years. We also asked participants about their acceptability of the tool using the Health Information Technology Usability Evaluation Scale (Health-ITUES). <h3>Setting</h3> Participants were living in the community setting. All participants (clinicians and people with history of concussion) completed an online survey. Representative clinicians and people with concussion participated in a semi-structured interview via Zoom. <h3>Participants</h3> A total of 16 participants were included in this study, comprising of 8 people with concussion and 8 healthcare providers. <h3>Interventions</h3> N/A. <h3>Main Outcome Measures</h3> Qualitative data (semi-structured interviews) and quantitative questionnaire (Health Information Technology Usability Evaluation Scale (Health-ITUES)) was used to understand acceptance and perception of the MyGuide Tool for concussion management. <h3>Results</h3> Participants reported two key benefits of using the online self-management tool, namely, the tool's emphasis on the interconnectedness of physical and psychological symptoms, and the ability to provide reassurance that symptoms being experienced were a normal part of the concussion experience. Clinicians described the tool as being useful as a supplementary source of information for clients in addition to clinical sessions, and believed the content was useful for increasing clients' independence in managing their own recovery. People with concussion indicated that the using the tool early in their recovery was important as knowledge and information was most needed at that point. Some people with concussion described that the tool had reduced benefit as it was used too late in their concussion management. Rather than using the site regularly as a management tool, several people with concussion reported that they worked through the material once and didn't refer to it afterwards. <h3>Conclusions</h3> Findings will provide insight into the acceptability and potential benefit of an online self-management tool in concussion rehabilitation and may provide important design considerations in this population. <h3>Author(s) Disclosures</h3> N/A.

Community-Based, Cluster-Randomized Pilot Trial of a Cardiovascular Mobile Health Intervention: Preliminary Findings of the FAITH! Trial.

Background:African Americans continue to have suboptimal cardiovascular health (CVH) based on the American Heart Association Life’s Simple 7 (LS7), 7 health-promoting behaviors and biological risk factors (eg, physical activity, blood pressure). Innovative, community-level interventions in partnership with trusted institutions such as African American churches are potential means to improve CVH in this population.Methods:Using a community-based participatory research approach, the FAITH! Trial (Fostering African American Improvement in Total Health) rigorously assessed the feasibility and preliminary efficacy of a refined, community-informed, mobile health intervention (FAITH! App) for promoting CVH among African Americans in faith communities using a cluster randomized controlled trial. Participants from 16 churches in Rochester and Minneapolis-St Paul, MN, were randomized to receive the FAITH! App (immediate intervention) or were assigned to a delayed intervention comparator group. The 10-week intervention core features included culturally relevant and LS7-focused education modules, diet/physical activity self-monitoring, and a group sharing board. Data were collected via electronic surveys and health assessments. Primary outcomes were average change in mean LS7 score (continuous measure of CVH ranging from poor to ideal [0–14 points]) from baseline to 6 months post-intervention (using generalized estimating equations) and app engagement/usability (by the Health Information Technology Usability Evaluation Scale; range, 0–5).Results:Of 85 enrolled participants (randomized to immediate [N=41] and delayed [control] intervention [N=44] groups), 76 and 68 completed surveys/health assessments at baseline and 6 months post-intervention, respectively (80% retention rate with assessments at both baseline and 6-month time points); immediate intervention [N=30] and control [N=38] groups). At baseline, the majority of participants (mean age [SD], 54.2 [12.3] years, 71% female) had <4-year college education level (39/66, 59%) and poor CVH (44% in poor category; mean LS7 score [SD], 6.8 [1.9]). The mean LS7 score of the intervention group increased by 1.9 (SD 1.9) points compared with 0.7 (SD 1.7) point in the control group (both P<0.0001) at 6 months. The estimated difference of this increase between the groups was 1.1 (95% CI, 0.6–1.7; P<0.0001). App engagement/usability was overall high (100% connection to app; >75% completed weekly diet/physical activity tracking; Health Information Technology Usability Evaluation Scale, mean [SD], 4.2 [0.7]).Conclusions:On the basis of preliminary findings, the refined FAITH! App appears to be an efficacious mobile health tool to promote ideal CVH among African Americans.Registration:URL: https://www.clinicaltrials.gov; Unique identifier: NCT03777709.

Combining human and machine intelligence for clinical trial eligibility querying.

ObjectiveTo combine machine efficiency and human intelligence for converting complex clinical trial eligibility criteria text into cohort queries.Materials and MethodsCriteria2Query (C2Q) 2.0 was developed to enable real-time user intervention for criteria selection and simplification, parsing error correction, and concept mapping. The accuracy, precision, recall, and F1 score of enhanced modules for negation scope detection, temporal and value normalization were evaluated using a previously curated gold standard, the annotated eligibility criteria of 1010 COVID-19 clinical trials. The usability and usefulness were evaluated by 10 research coordinators in a task-oriented usability evaluation using 5 Alzheimer’s disease trials. Data were collected by user interaction logging, a demographic questionnaire, the Health Information Technology Usability Evaluation Scale (Health-ITUES), and a feature-specific questionnaire.ResultsThe accuracies of negation scope detection, temporal and value normalization were 0.924, 0.916, and 0.966, respectively. C2Q 2.0 achieved a moderate usability score (3.84 out of 5) and a high learnability score (4.54 out of 5). On average, 9.9 modifications were made for a clinical study. Experienced researchers made more modifications than novice researchers. The most frequent modification was deletion (5.35 per study). Furthermore, the evaluators favored cohort queries resulting from modifications (score 4.1 out of 5) and the user engagement features (score 4.3 out of 5).Discussion and ConclusionFeatures to engage domain experts and to overcome the limitations in automated machine output are shown to be useful and user-friendly. We concluded that human–computer collaboration is key to improving the adoption and user-friendliness of natural language processing.

Establishing a usability cut-point for the health information technology usability evaluation scale (Health-ITUES)

ObjectiveThe Health Information Technology Usability Evaluation Scale (Health-ITUES) is a validated and reliable instrument to evaluate usability of information technology (IT) tools. In this study, we aimed to establish the optimal cut-point of the Health-ITUES to identify usability of IT tools. MethodsAdult participants were recruited to a trial evaluating a mobile app for self-managing HIV. Participants completed the Health-ITUES at the 3- and 6-month follow-up. Health-ITUES is a 20-item questionnaire that assesses four subscales: impact, perceived usefulness, perceived ease of use, and user control. The total score ranged from 1 to 5 and a higher score indicates greater usability. App use was defined as the proportion of activities completed by participants in both study arms. The selection of an optimal cut-point involved a series of multiple linear regression models with 500 bootstrap replications to examine the relationship between the Health-ITUES total score and app use, controlling for potential covariates. ResultsWe included 158 participants; mean age was 49.7 years (SD 10.3), 71% were African American/Black, and 72% were non-Hispanic. Mean Health-ITUES total scores at 3 and 6 months were 4.39 (SD 0.75) and 4.43 (SD 0.75), respectively. App use completedby participants from baseline to the 3-month follow-up visits was 0.61 (SD 0.36, range 0–1.72) and from 3-month to the 6-month follow-up visits was 0.51 (SD 0.37). Participants who reported greater Health-ITUES total score completed more activities [β = 0.18, 95% Confidence Interval (CI) 0.10–0.27]. The optimal cut-point of 4.32 (95% CI: 4.25–4.56) yielded the lowest p-value to identify usability of IT tools. ConclusionsIn this study of adults with HIV, we identified an optimal cut-point of 4.32 on the Health-ITUES total score to define usability. Further studies are needed to validate this cut-point.

Validity and Reliability of the Korean Version of the Health Information Technology Usability Evaluation Scale: Psychometric Evaluation.

BackgroundRigorous development of mobile technologies requires the use of validated instruments to evaluate the usability of these tools, which has become more relevant with the expansion of these technologies. Although various usability evaluation tools have been developed, there are relatively few simple evaluation instruments that have been validated across diseases and languages in mobile health (mHealth) information technology for use in multiple diseases.ObjectiveThe purpose of this study is to validate the Korean version of the Health Information Technology Usability Evaluation Scale (Korean Health-ITUES) and assess its applicability for different health conditions.MethodsTo develop the Korean Health-ITUES, we used a validation process involving the following 3 steps: (1) customization of the Health-ITUES for menstrual symptoms, (2) translation of the Health-ITUES from English into Korean, and (3) examination of the reliability and validity of the instrument. The translation process adhered to the World Health Organization (WHO) guidelines for translation and back-translation, expert review, and reconciliation.ResultsThe Korean Health-ITUES showed reliable internal consistency with Cronbach α=.951; meanwhile, factor loadings of the 20 items in the 4 subscales ranged from 0.416 to 0.892.ConclusionsThe Health-ITUES demonstrated reliability and validity for its use in assessing mHealth apps’ usability in young Korean women with menstrual discomfort. Given the strong psychometric properties of this tool in Korean and English and across 2 different health conditions, the Health-ITUES is a valid and reliable instrument for assessing the usability of mHealth apps. The Health-ITUES is also a valid instrument for evaluating mHealth technologies, which are widely used by patients to self-manage their health and by providers to improve health care delivery.

Sign of the times: Community engagement to refine a cardiovascular mHealth intervention through a virtual focus group series during the COVID-19 Pandemic

BackgroundAfrican-Americans are underrepresented in mobile health intervention research studies which can perpetuate health inequities and the digital divide. A community-based, user-centered approach to designing mobile health interventions may increase their sociocultural relevance and effectiveness, especially with increased smartphone use during the coronavirus disease 2019 pandemic. We aimed to refine an existing mobile health intervention via a virtual focus group series.MethodsAfrican-American community members (n = 15) from churches in Minneapolis-St. Paul and Rochester, Minnesota were enrolled in a virtual (via videoconferencing), three-session focus group series over five months to refine a cardiovascular health-focused mobile health application (FAITH! [Fostering African-American Improvement in Total Health!] App). Participants accessed the app via their smartphones and received a Fitbit synced to the app. Participants engaged with multimedia cardiovascular health-focused education modules, a sharing board for social networking, and diet/physical activity self-monitoring. Participant feedback on app features prompted iterative revisions to the FAITH! App. Primary outcomes were app usability (assessed via Health Information Technology Usability Evaluation Scale range: 0–5) and user satisfaction.ResultsParticipants (mean age [SD]: 56.9 [12.3] years, 86.7% female) attended a mean 2.8 focus groups (80% attended all sessions). The revised FAITH! App exceeded the goal Health Information Technology Usability Evaluation Scale score threshold of ≥4 (mean: 4.39, range: 3.20–4.95). Participants positively rated updated app content, visual appeal, and use of social incentives to maintain engagement. Increasing user control and refinement of the moderated sharing board were identified as areas for future improvement.ConclusionsCommunity-partnered, virtual focus groups can optimize usability and increase participant satisfaction of mobile health lifestyle interventions that aim to promote cardiovascular health in African-Americans.

Tools for Evaluating the Content, Efficacy, and Usability of Mobile Health Apps According to the Consensus-Based Standards for the Selection of Health Measurement Instruments: Systematic Review.

BackgroundThere are several mobile health (mHealth) apps in mobile app stores. These apps enter the business-to-customer market with limited controls. Both, apps that users use autonomously and those designed to be recommended by practitioners require an end-user validation to minimize the risk of using apps that are ineffective or harmful. Prior studies have reviewed the most relevant aspects in a tool designed for assessing mHealth app quality, and different options have been developed for this purpose. However, the psychometric properties of the mHealth quality measurement tools, that is, the validity and reliability of the tools for their purpose, also need to be studied. The Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) initiative has developed tools for selecting the most suitable measurement instrument for health outcomes, and one of the main fields of study was their psychometric properties.ObjectiveThis study aims to address and psychometrically analyze, following the COSMIN guideline, the quality of the tools that are used to measure the quality of mHealth apps.MethodsFrom February 1, 2019, to December 31, 2019, 2 reviewers searched PubMed and Embase databases, identifying mHealth app quality measurement tools and all the validation studies associated with each of them. For inclusion, the studies had to be meant to validate a tool designed to assess mHealth apps. Studies that used these tools for the assessment of mHealth apps but did not include any psychometric validation were excluded. The measurement tools were analyzed according to the 10 psychometric properties described in the COSMIN guideline. The dimensions and items analyzed in each tool were also analyzed.ResultsThe initial search showed 3372 articles. Only 10 finally met the inclusion criteria and were chosen for analysis in this review, analyzing 8 measurement tools. Of these tools, 4 validated ≥5 psychometric properties defined in the COSMIN guideline. Although some of the tools only measure the usability dimension, other tools provide information such as engagement, esthetics, or functionality. Furthermore, 2 measurement tools, Mobile App Rating Scale and mHealth Apps Usability Questionnaire, have a user version, as well as a professional version.ConclusionsThe Health Information Technology Usability Evaluation Scale and the Measurement Scales for Perceived Usefulness and Perceived Ease of Use were the most validated tools, but they were very focused on usability. The Mobile App Rating Scale showed a moderate number of validated psychometric properties, measures a significant number of quality dimensions, and has been validated in a large number of mHealth apps, and its use is widespread. It is suggested that the continuation of the validation of this tool in other psychometric properties could provide an appropriate option for evaluating the quality of mHealth apps.

Remote System for Daily Symptom Monitoring During Systemic Anticancer Treatment

Electronic systems for remotely monitoring symptoms during cancer treatment are increasingly being used. Most of them are intended for weekly or periodic symptom monitoring. The aim of this study was to explore the feasibility and usability of a remote system for daily symptom monitoring during systemic anticancer treatment. We offered a remote system for daily symptom monitoring to patients starting their first ever systemic therapy. Patient acceptance was observed as the proportion of patients showing interest in using the system. System users were invited to complete the "Health information Technology Usability Evaluation Scale," and the number of patients' self-reports was used to evaluate patient compliance. Of 465 patients, 239 (51.4%) showed interest in using the system; 111 system users reported a fairly good overall usability score (4 of 5), and perceived ease of use scored the highest (4.7 of 5) and perceived usefulness lowest (3.7 of 5). Their daily compliance was 68.8% (±31.1) at 3 weeks and 59.1% (±31.2) at 12 weeks after the start of treatment. Less than half of respondents (49.5%) and only one third (31.2%) perceived that their symptom reports were used by the nurses and doctors, respectively. Half of patients starting their first ever systemic treatment showed interest in using a remote system for daily symptom monitoring. For these patients, daily symptom monitoring seems acceptable, and the system is well complied within the first 12 weeks of treatment. A remote system for daily symptom monitoring is feasible and enables quick response to deteriorating symptoms. Use by healthcare professionals is a point of improvement.

Usability Evaluation of the mLab App for Improving Home HIV Testing Behaviors in Youth at Risk of HIV Infection.

Improving access to HIV testing among youth at high risk is essential for reaching those who are most at risk for HIV and least likely to access health care services. This study evaluates the usability of mLab, an app with image-processing feature that analyzes photos of OraQuick HIV self-tests and provides real-time, personalized feedback. mLab includes HIV prevention information, testing reminders, and instructions. It was developed through iterative feedback with a youth advisory board (N = 8). The final design underwent heuristic (N = 5) and end-user testing (N = 20). Experts rated mLab following Nielsen's heuristic checklist. End-users used the Health Information Technology Usability Evaluation Scale. While there were some usability problems, overall study participants found mLab useful and user-friendly. This study provides important insights into using a mobile app with imaging for interpreting HIV test results with the goal of improving HIV testing and prevention in populations at high risk.

A Uniquely Targeted, Mobile App-Based HIV Prevention Intervention for Young Transgender Women: Adaptation and Usability Study.

BackgroundYoung transgender women (YTW) are a key population for HIV-related risk reduction, yet very few interventions have been developed to meet their needs. Mobile health interventions with the potential for both efficacy and wide reach are a promising strategy to reduce HIV risk among YTW.ObjectiveThis study aims to adapt an efficacious group-based intervention to a mobile app, Project LifeSkills, to reduce HIV risk among YTW, and to test its acceptability and usability.MethodsThe group-based intervention was adapted to a mobile app, LifeSkills Mobile, with input from an expert advisory group and feedback from YTW collected during user-centered design sessions. A beta version of the app was then tested in a usability evaluation using a think-aloud protocol with debriefing interviews, recordings of screen activity, and assessments of usability via the Post-Study System Usability Questionnaire (PSSUQ) and the Health Information Technology Usability Evaluation Scale (Health-ITUES).ResultsYTW (n=8; age: mean 24 years, SD 3 years; racial or ethnic minority: 7/8, 88%) provided feedback on the app prototype in design sessions and then tested a beta version of the app in a usability trial (n=10; age: mean 24 years, SD 3 years; racial or ethnic minority: 8/10, 80%). Both usability ratings (Health-ITUES: mean 4.59, SD 0.86; scale range: 1-5) and ratings for satisfaction and accessibility (PSSUQ: mean 4.64, SD 0.90; scale range 1-5) were in the good to excellent range. No functional bugs were identified, and all mobile activities were deployed as expected. Participant feedback from the usability interviews indicated very good salience of the intervention content among the focal population. Participants’ suggestions to further increase app engagement included adding animation, adding audio, and reducing the amount text.ConclusionsWe conclude that the LifeSkills Mobile app is a highly usable and engaging mobile app for HIV prevention among YTW.