User Experience Evaluation Research Articles

A user experience evaluation of a mobile application for disseminating site‐specific impact‐based flood warnings: The A4alerts app

AbstractPeople‐centred early warning systems must ensure that their warnings are timely, actionable and get to the largest number of exposed individuals in the shortest time possible. Due to the increased access to high‐bandwidth communication, mobile applications have become a valuable complement to traditional sources of emergency information. However, there is limited research on how useful end‐users perceive these apps and their functionalities in the context of flood emergencies. This research presents the first online user experience evaluation of the A4alerts, a crisis app developed to disseminate the flood warnings triggered by the operational impact‐based early warning system in Catalonia, Spain. Following an interactive experiment using the A4alerts app to access emergency information within real‐based flood scenarios, participants were prompted to measure their experience using statements linked to criteria on usefulness, functional appropriateness, and adoption. The evaluation results suggest that sampled participants considered the A4alerts a useful communication tool to support end‐users and their emergency actions. In particular, the local impact‐based flood warnings, regional official warnings and self‐protection action checklist were ranked as the most useful functionalities implemented within the app. These findings justify future iterations of the A4alerts for supporting the rapid dissemination of emergency information and helping communities reduce the impacts of flood events.

User Experience Evaluation of Upper Limb Rehabilitation Robots: Implications for Design Optimization: A Pilot Study.

With the development of science and technology, people are trying to use robots to assist in stroke rehabilitation training. This study aims to analyze the result of the formative test to provide the orientation of upper limb rehabilitation robot design optimization. We invited 21 physical therapists (PTs) and eight occupational therapists (OTs) who had no experience operating any upper limb rehabilitation robots before, and 4 PTs and 1 OT who had experience operating upper limb rehabilitation robots. Data statistics use the Likert scale. The general group scored 3.5 for safety-related topics, while the experience group scored 4.5. In applicability-related questions, the main function score was 2.3 in the general group and 2.4 in the experience group; and the training trajectory score was 3.5 in the general group and 5.0 in the experience group. The overall ease of use score was 3.1 in the general group and 3.6 in the experience group. There was no statistical difference between the two groups. The methods to retouch the trajectory can be designed through the feedback collected in the formative test and gathering further detail in the next test. Further details about the smooth trajectory must be confirmed in the next test. The optimization of the recording process is also important to prevent users from making additional effort to know it well.

Taxonomy and definition of audio augmented reality (AAR): A grounded theory study

AR applications have mostly considered visual augmentations while excluding other modalities. Recent developments in audio augmented reality (AAR) applications have been based on the definitions of visual AR or mixed reality (MR), and thus, AAR technology development has lacked systematic efforts. We investigated the concept of augmented reality through audio to provide a systematic understanding and generate a taxonomy and a definition for AAR. A conference workshop (N = 28), focus groups (N = 18), and expert interviews (N = 6) generated qualitative data regarding the concept of AAR. Grounded Theory (GT) was used to analyze the data and produce a new taxonomy and a definition. The AAR taxonomy consists of three categories – Environment Connected, Goal Directed, and Context Adapted, with three subcategories respectively. The need for a separate taxonomy for AAR is highlighted to aid in the development of AAR applications in a systematic manner. The taxonomy is expected to be used as a heuristic tool that can guide developers to build AAR applications and can be used in evaluating user experience with AAR applications.

Educational Seismology through an Immersive Virtual Reality Game: Design, Development and Pilot Evaluation of User Experience

Virtual Reality (VR), especially in its immersive form, is a promising technology employed to support education and training in various fields. VR offers unique opportunities to experience situations and environments that are otherwise inaccessible or risky. Educational Seismology aims to inform and educate the public on earthquakes; to this end, the use of VR is investigated as an attractive solution. VRQuake is an immersive VR application designed and developed for Educational Seismology purposes. VRQuake is structured in five consecutive scenes and is organized as a game. It allows users to interact with virtual objects in real time and apply learned rules and good practices in reaction to an earthquake, thus providing a dynamic learning environment. A pilot evaluation of VRQuake is performed by volunteer university students who play the game and then answer a questionnaire with closed- and open-type questions referring mostly to the user experience. Analysis of the answers has shown positive results regarding usability, clarity and acceptance of the application. Answers are also encouraging as to the educational potential of VRQuake. Furthermore, qualitative analysis of open-type questions has contributed user suggestions and demands that point to interesting new directions for further improvement of user experience and learning outcomes.

Evaluasi User Engagement dan User Experience pada Aplikasi Satu Sehat

Various efforts to prevent and trace the wider spread of the Covid-19 Virus have been carried out by the government. One of them is done with through technology by providing the application namely SatuSehat application which has variety of features aimed on helping the society. However, along with the imminent evolution of this application into SatuSehat application, these various features that exist, are still rarely known by the public. In addition, from previous studies, there are user experience evaluation has been carried out which shows that low evaluation results. Even though this application has been widely downloaded by the public. This study is aimed at re-measuring the user experience of SatuSehat application and is accompanied by user engagement evaluation to SatuSehat application and its relationship with the user experience. User experience evaluation will utilize the UEQ-S method while the user engagement evaluation will utilize the UEQ-SF method. The results showed that SatuSehat application has a positive overall user experience evaluation result which is supported by quite good user engagement evaluation results. The results of the study even show that there is a strong correlation between user engagement and user experience from the SatuSehat application.

A co-produced service evaluation of ethnic minority community service user experiences of a specialist mental health service during the COVID-19 pandemic

BackgroundFor ethnic minority communities in the UK, the COVID-19 pandemic amplified existing health inequalities and created other consequential disadvantages like increased vulnerability to COVID-19, higher rates of hospital admissions, increased mortality and poorer mental health outcomes. While longer-term impacts of COVID-19 are considered, it is crucial for NHS mental health services to understand the specific barriers and needs of ethnic minority communities to provide consistent and equitable access to mental health services. These aspects were the focus of a service evaluation of a Sussex-wide mental health service conducted in co-production with experts-by-experience, public members, health professionals and researchers from ethnic minority communities.MethodsCo-designed creative workshops (n = 13) and semi-structured qualitative interviews (n = 13) were used to explore experiences of accessing specialist mental health services during the COVID-19 pandemic. Participants were: Sussex Partnership NHS Foundation Trust (SPFT) service users recruited between October 2021 and January 2022; aged 16+; from ethnic minority community backgrounds. Data was analysed using Thematic Analysis.ResultsThe analysis yielded five overarching themes contextualising service users’ experiences: (1) limited awareness of SPFT mental health services; (2) effects of COVID-19 in gaining access to SPFT; (3) SPFT reaching out to ethnic minorities; (4) being supported, 4a) hiding my mental health status from friends and families, 4b) lack of ethnic diversity in services, and 4c) better provision of information and support services, (5) relationship between childhood experiences and current mental health. These findings led to seven key recommendations for future service developments within SPFT.ConclusionsAlthough this evaluation was set in the context of COVID-19, findings have highlighted specific mental health service needs for ethnic minorities that are applicable beyond the confines of the pandemic. Many benefited from online sessions seen as more inclusive. Mental health advocates, outreach and joint working with communities could help further reduce stigmatising attitudes and improve engagement with mental health services. Improved service awareness of the impact of childhood or historical traumas experienced by ethnic minority communities on current mental health, the role of cultural awareness training and availability of culturally adapted therapies is also needed. Many service improvement recommendations provided could impact all service users.

Different Frameworks, Similar Results? Head-to-Head Comparison of the Generic Preference-Based Health-Outcome Measures CS-Base and EQ-5D-5L.

We compared two generic, preference-based health-outcome measures: the novel patient-centered Château-Santé Base (CS-Base), entailing a multi-attribute preference response framework, and the widely used EQ-5D-5L, regarding effects of different measurement frameworks and different descriptive systems. We conducted a cross-sectional study using a random sample of patients (3019 reached, 1988 included) in the USA with various health conditions. The CS-Base (12 attributes, each with four levels), EQ-5D-5L and the 5D-4L (an ad hoc, multi-attribute preference response-based measure that includes five attributes similar to the EQ-5D-5L, but with four levels) were used as health-outcome measures. We compared the proportions of problems reported on health attributes, statistical robustness and face validity of coefficients, attribute importance, differentiation between health states based on health-state values obtained with these measures, and user experience. All the CS-Base and 5D-4L coefficients had logical orders and significant differences from the reference level (p < 0.001). Substantial differences were observed in the CS-Base and 5D-4L coefficients between all levels on all attributes, while subtle differences were seen in those of the EQ-5D-5L. Attribute importance of usual (daily) activities were lowest or second lowest in all the three health-outcome measures. Attributes with the highest importance in the CS-Base, 5D-4L, and EQ-5D-5L were respectively mobility, anxiety/depression, and pain/discomfort. Four attributes are similar between the CS-Base and EQ-5D-5L, eight are exclusive to CS-Base. Of the eight, vision and hearing had the highest importance. Health-state values showed a smoother distribution with minimal discontinuity in the CS-Base and EQ-5D-5L than in the 5D-4L. In user experience evaluation, both CS-Base and the 5D-4L showed mean scores above 50 (indicating positive evaluation) in terms of the description of health and ease of understanding. This study demonstrated that CS-Base and 5D-4L, which are grounded in the multi-attribute preference response framework, produced statistically robust coefficients, with better face validity than those for the EQ-5D-5L. CS-Base and the EQ-5D-5L outperformed the 5D-4L in differentiating between health states. Notwithstanding differences in content, measurement frameworks, and estimated coefficients, the computed health-state values were similar between CS-Base and EQ-5D-5L.

Uji UI UX Pada Aplikasi Peduli Lindungi Menggunakan User Experience Questionare (UEQ) di Gedung Perkantoran Gading Serpong

The application called Peduli Protect is a form of government policy to control and overcome various problems that arise due to the Covid-19 pandemic. The Peduli Protect application aims at three main priorities, namely 1) protecting the public in accessing public facilities, 2) tracing people who are indicated to be exposed to Covid-19, and 3) information on mapping the zoning of exposure to Covid-19. One of the testing tools for the Cares Protect Application is the User Experience Questionnaire (UEQ), where the User Experience Questionnaire (UEQ) is used to describe the experience gained by the user or UX for satisfaction and usability in the visual appearance of the application, both text boxes, drop-down lists, and other components or commonly called UI when using App Care Protect device. In this study, the method used is combined, namely qualitative quantitative (where technical data collection is done by distributing questionnaires, where each response to each statement item in the questionnaire has a different assessment area and respondents are also required to fill in their demographic data). The total population in this study is 100 Respondents, and the samples taken of all the total population are 100 respondents too. The formulations in this study are 1) How to evaluate the user experience/UX in using the Peduli Protect Application with User Experience Questionnaire (UEQ)? While the objectives of this study are 1) to determine the results of the evaluation of user experience/UX in using the Cares Protect Application using the User Experience Questionnaire (UEQ). The results of the evaluation of user experience / user experience / UX using User Experience Questionnaire (UEQ) in 6 aspects are attractiveness 0,06; clarity 0,53; efficiency 0,36; accuracy 0,17; stimulation 0,09; and novelty 0,01. This shows that all aspects of the user experience get a negative impression. The results of this study can be used as recommendations for designing the new Peduli Protect application user experience by innovating in various aspects.

EEG-Based Machine Learning Models to Evaluate Haptic Delay: Should We Label Data Based on Self-Reporting or Physical Stimulation?

Reliable haptic interfaces augment human-computer interaction via simulated tactile and kinesthetic feedback. As haptic technologies advance, user experience evaluation becomes more crucial. Conventionally, self-reporting is used to evaluate haptic experiences; however, it could be inconsistent or imprecise due to human error. A promising alternative is using neurocognitive methods with machine or deep learning models to evaluate the human haptic experience. Machine and deep learning models can be trained on Electroencephalography (EEG) data labeled based on self-report or actual physical stimulation. As the literature lacks a systematic study on which approach is more robust, we develop a visuo-haptic task to answer this question by examining an important haptic experience, namely, haptic delay. EEG is recorded during the experiment, and participants report whether they detected a delay in the haptic modality through self-report. Four machine/deep learning models were trained twice on the EEG data using the two labeling methods. Models trained with labels from the physical stimuli significantly outperformed those trained with self-reporting labels. Although this finding holds true for one particular haptic experience (haptic delay), it cannot be extrapolated to others; rather, it suggests that EEG data labeling plays a prominent role in evaluating the haptic experience through neurocognitive methods.

The effect of digital literacy on depressive symptoms among older Korean women: a mediation analysis focusing on the role of social support.

The purpose of this study was to investigate the relationship between digital literacy and depressive symptoms, as well as the mediating role of social support in this relationship, among older women (60 years and older) in Korea. This study analyzed data from the User Experience Evaluation Survey, which was conducted by the Ewha Institute for Age Integration Research to improve the accessibility of digital information for older adults research from May to September 2020. Survey data on depressive symptoms, digital literacy, and social support were analyzed using descriptive statistics, Pearson correlation coefficients, and multiple regression. The factors influencing depressive symptoms among older women included work status (B=-.19, p=.01), social support (B=-.17, p<.001), self-rated health (B=-.13, p=.003), and digital literacy (B=-.10, p=.005), which had an explanatory power of 33%. In addition, social support played a mediating role in the relationship between digital literacy and depressive symptoms (B= -.05, SE=.02; 95% CI, -.09 to -.02). The findings of this study support the need to develop and apply interventions that promote digital literacy among older women to mitigate depressive symptoms by increasing social support.

Evaluasi Pengalaman Pengguna Aplikasi Hiip Indonesia Menggunakan Metode USE Questionnaire dan Usability Testing

With the rapid development of mobile applications, user experience has become crucial in ensuring success and user satisfaction. This research paper presents an evaluation of the user experience of the Hiip Indonesia application, with a focus on usability assessment and user satisfaction. The research methodology employs two main methods: the USE Questionnaire and Usability Testing. The USE Questionnaire is a validated and widely used tool for assessing user satisfaction, while Usability Testing involves observing user interactions with the application and collecting qualitative and quantitative data. The research findings indicate that the usability level of the Hiip Indonesia application falls into the "acceptable" category, with a total feasibility percentage of 73.91%. The hypothesis testing shows a significant influence on satisfaction between the variables of usefulness, ease of use, and ease of learning. However, on a partial level, only the variables of usefulness and ease of use significantly influence satisfaction. In contrast, the ease of learning variable does not have a considerable impact. Furthermore, the user experience evaluation data through task scenario testing indicates that the Hiip Indonesia application is highly effective to use, with a task completion success rate of 100% and an average task completion time of approximately 46.77 seconds. After reviewing the identified issues, recommendations for improvements to the Hiip Indonesia application have been formulated. Some of these recommendations include adding features to the "campaign" menu, adding a menu to the home screen, adding features to the "social media" menu, and adding features for chatting with administrators.&#x0D; &#x0D; Keywords: User experience, UX evaluation, USE Questionnaire, usability testing, mobile application, PT. Hiip Inovasi Indonesia.

Development and Validation of Four Social Scales for the UX Evaluation of Interactive Products

The social dimension of interactive products covers all aspects of our relationships with others that are impacted by owning and using such products. Although social features are making their way into a growing number of interactive products, there is a lack of an evaluation tool to capture the social dimension of the user experience (UX). This study addressed this shortcoming by developing and validating new social scales based on the UEQ + framework. We developed four social scales to encompass various aspects within the social dimension. For scale development, 229 participants rated their UX with products having social aspects. Exploratory factor analysis allowed us to identify four sub-dimensions (Identification, Social interaction, Social stimulation, and Social acceptance), each evaluated with four items. For scale validation, 450 participants evaluated the UX of three product categories, using the new social scales, AttrakDiff, and the six UX dimensions of UEQ+. Results of MANOVA showed that the social scales discriminated the three categories (F (8, 560) = 20.68, p < 0.001, Pillai’s trace = 0.456). The four social scales developed in this study can be combined with other UX dimensions of the UEQ + modular framework to provide a comprehensive overview of user interaction with products.

DrivingVibe: Enhancing VR Driving Experience using Inertia-based Vibrotactile Feedback around the Head

We present DrivingVibe, which explores vibrotactile feedback designs around the head to enhance VR driving motion experiences. We propose two approaches that use a 360-degree vibrotactile headband: 1) mirroring and 2) 3D inertia-based. The mirroring approach extends the vibrotactile patterns of handheld controllers to actuate the entire headband uniformly. The 3D inertia-based approach uses the acceleration telemetry data that driving games/simulators export to motion platforms to generate directional vibration patterns, including: i) centrifugal forces, ii) horizontal acceleration/deceleration, and iii) vertical motion due to rough terrain. The two approaches are complementary as the mirroring approach supports all driving games because it does not require telemetry data, while the 3D inertia-based approach provides higher feedback fidelity for games that provide such data. We conducted a 24-person user experience evaluation in both passive passenger mode and active driving mode. Study results showed that both DrivingVibe designs significantly improved realism, immersion, and enjoyment (p&lt;.01) with large effect sizes for the VR driving experiences. For overall preference, 88% (21/24) of participants preferred DrivingVibe, with a 2:1 preference for 3D inertia-based vs. mirroring designs (14 vs. 7 participants). For immersion and enjoyment, 96% (23/24) of participants preferred DrivingVibe, with nearly a 3:1 preference (17 vs. 6 participants) for the 3D inertia-based design.

SensCon: Embedding Physiological Sensing into Virtual Reality Controllers

Virtual reality experiences increasingly use physiological data for virtual environment adaptations to evaluate user experience and immersion. Previous research required complex medical-grade equipment to collect physiological data, limiting real-world applicability. To overcome this, we present SensCon for skin conductance and heart rate data acquisition. To identify the optimal sensor location in the controller, we conducted a first study investigating users' controller grasp behavior. In a second study, we evaluated the performance of SensCon against medical-grade devices in six scenarios regarding user experience and signal quality. Users subjectively preferred SensCon in terms of usability and user experience. Moreover, the signal quality evaluation showed satisfactory accuracy across static, dynamic, and cognitive scenarios. Therefore, SensCon reduces the complexity of capturing and adapting the environment via real-time physiological data. By open-sourcing SensCon, we enable researchers and practitioners to adapt their virtual reality environment effortlessly. Finally, we discuss possible use cases for virtual reality-embedded physiological sensing.

Analisis Perbandingan User Experience pada Website dan Aplikasi Mobile Shopee Menggunakan UEQ

In the midst of technological developments that are always advancing, the internet is one of the things that makes life easier. The internet itself produces products that are useful for daily activities, for example websites and applications. These two products have many types of usage categories, one of which is e-commerce. E-commerce (electronic commerce) provides a different shopping experience for consumers. One of the e-commerce that is developing in Indonesia is Shopee. Shopee has 2 (two) versions, a mobile application and a website. In this case, of course, the layout of the user interface provided is definitely different. Different layouts certainly provide a different user experience. The purpose of this research is to analyze and evaluate user experience and to find out user perceptions and interaction problems experienced by users when using mobile applications and the Shopee website. This study uses 2 (two) research methods, namely user experience questionnaire and Focus Group Discussion. The results of this analysis show that the mobile application and Shopee website have significant values ​​for each parameter. Both Shopee platforms only have 4 (four) of the 6 (six) parameters that reach 0.8. This shows that the Shopee mobile application and the Shopee website have their own strengths and weaknesses in terms of user experience.

Evaluation of User Experience (UX) Design for Emerging Technologies

The unique challenges inherent in designing for emerging technologies. As these technologies often lack established design principles, UX designers are confronted with the task of navigating uncharted territory. This section examines the complexities associated with understanding user behaviors, expectations, and preferences when interacting with cutting-edge technologies. Adaptive design, user-centered approaches, and iterative prototyping are among the strategies discussed to ensure a user-centric approach that accommodates the evolving nature of these technologies. Special attention is given to the integration of human-computer interaction (HCI) theories and cognitive psychology principles to enhance the overall user experience. UX design for emerging technologies places humans at the center of technological innovation. Research in this area ensures that new technologies align with user needs, preferences, and behaviors, fostering a more natural and intuitive interaction between users and technology. The dynamic nature of emerging technologies necessitates continuous adaptation. Research in UX design provides insights into evolving user expectations, enabling designers and developers to future-proof their practices. This adaptability ensures that design solutions remain relevant and effective as technologies advance. The COPRAS-G method requires identifying selection criteria; evaluating information related to these criteria, and developing methods to evaluate Meeting the participant's needs Criteria for doing in order to assess the overall performance of the surrogate. Decision analysis involves a Decision Maker (DM) Situation to do consider a particular set of alternatives and select one among several alternatives, usually with conflicting criteria. For this reason, the developed complexity proportionality assessment (COPRAS) method can be used. From the result Voice Assistant is got the first rank whereas is the VR headset is having the lowest rank.

Augmented Reality Order Picking Aid for Foreign Workers in Warehouses

Background: In warehouse logistics, picking work comprises roughly 60% of the costs, emphasizing the need for efficiency (Matsumoto et al., 2019). Traditionally, workers utilize written instructions, which can be challenging for those less language proficient. The integration of augmented reality (AR) head- mounted displays (HMDs) may enhance accuracy and efficiency (Matsumoto et al., 2019). However, AR HMDs may come with some drawbacks—after extended use, some users report discomfort and decreased performance (Vidovič &amp; Gajšek, 2020). Despite promising improvements in warehouse operations, HMDs’ long-term impacts remain uncertain (Fang et al., 2019). Furthermore, the influence of language proficiency on HMD effectiveness needs exploration (Murauer et al, 2018). In order to fully comprehend the potential and limitations of HMDs, further research is necessary, targeting effective strategies for implementation and optimal AR user interface (UI) design. Objective: This study aims to evaluate an AR HMD system against traditional methods, focusing on its potential to aid non-native English-speaking warehouse workers and boost efficiency and accuracy in picking tasks. Our goal is to ascertain whether an AR aid system, utilizing universal and conformal design principles, can yield superior results in user performance, usability, and situational awareness compared to written instructions. Method: We identified the language-related challenges faced by foreign workers through interviews. Guided by these insights and Ganapathy’s mobile AR guidelines (Ganapathy et al., 2013), we designed an AR solution with universal symbols and intuitive interactions. The AR solution was prototyped using Microsoft HoloLens 2. To evaluate user experience with the proposed system, we conducted a within-subject experiment in a controlled laboratory environment, comparing this AR headset instruction with traditional written instructions. We employed the situation awareness rating technique (SART) questionnaire (Taylor et al., 2017) and system usability scale (SUS) questionnaire (Brooke et al., 1995), along with performance measures, to assess the effectiveness of the proposed system. Results: Our study with 17 participants indicated no significant difference in task completion time between traditional and AR headset instructions. However, AR significantly reduced package identification time ( M=6.89, SE=0.40 vs. M=10.15, SE=0.61). Moreover, people with the AR instructions had no errors while with the traditional written instructions had a total of 2 errors. The proposed AR aid also resulted in enhanced worker situation awareness by allowing them not to divide their attention across job instructions and the dynamic warehouse environment ( M=2.41, SE=0.24 vs. M=3.70, SE=0.35). The AR headset was perceived as easier to use ( M=4.35, SE=0.16 vs. M=3.35, SE=0.17) and better integrated various functions ( M=3.94, SE=0.16 vs. M=2.70, SE=0.19), despite some participants reported a need for technical assistance. Conclusion: The human-subject experiment demonstrates that the proposed AR aid system is effective in eliminating errors, improving ease of use, and enhancing situation awareness of foreign workers in warehouses. This study also underscores the importance of a user-centered approach in leveraging technology for users in diverse contexts. Application: Our proposal holds promising prospects beyond the scope of the study. Its potential extends to various safety- critical domains, including transportation, construction, and military operations, where operators’ awareness of the dynamic environment is crucial (Kim et al., 2020).

User Experience Evaluation of a Virtual Reality Tool Used for 3D Modelling in Industrial Design Education: A Study in the Indian Context

Virtual reality (VR) technology has recently been adopted by educators for use in the classroom. Currently, this educational model includes not only lectures with teachers in the online classroom but also practical sessions using online platforms. Few studies have explored the potential of pedagogical approaches to implementing VR in the classroom for the purpose of design education. The focus of this paper was to study the learning experiences of the 3D visualisation of products among industrial design students through the strategic implementation of virtual reality technology. A within-subjects comparative study was conducted to measure cognitive workload and engagement and enjoyment, while a 3D modelling task was given using two different set-ups (conventional 3D software versus VR-based software). The statistical results show that the NASA-TLX score was lower in the case of the VR-based 3D modelling exercise compared to the conventional 3D software-based exercise. On the other hand, the mean values were higher for the engagement and enjoyment and usability scores, which means that the VR-based experience for 3D modelling was better than the traditional modelling experience using conventional software. Hence, there are possibilities to implement VR-based 3D modelling tools for online industrial design education for 3D visualisation in the near future.

A Digital Counselor-Delivered Intervention for Substance Use Among People With HIV: Development and Usability Study.

Substance use disorders are prevalent and undertreated among people with HIV. Computer-delivered interventions (CDIs) show promise in expanding reach, delivering evidence-based care, and offering anonymity. Use in HIV clinic settings may overcome access barriers. Incorporating digital counselors may increase CDI engagement, and thereby improve health outcomes. We aim to develop and pilot a digital counselor-delivered brief intervention for people with HIV who use drugs, called "C-Raven," which is theory grounded and uses evidence-based practices for behavior change. Intervention mapping was used to develop the CDI including a review of the behavior change research in substance use, HIV, and digital counselors. We conducted in-depth interviews applying the situated-information, motivation, and behavior skills model and culturally adapting the content for local use with people with HIV. With a user interaction designer, we created various digital counselors and CDI interfaces. Finally, a mixed methods approach using in-depth interviews and quantitative assessments was used to assess the usability, acceptability, and cultural relevance of the intervention content and the digital counselor. Participants found CDI easy to use, useful, relevant, and motivating. A consistent suggestion was to provide more information about the negative impacts of drug use and the interaction of drug use with HIV. Participants also reported that they learned new information about drug use and its health effects. The CDI was delivered by a "Raven," digital counselor, programmed to interact in a motivational interviewing style. The Raven was perceived to be nonjudgmental, understanding, and emotionally responsive. The appearance and images in the intervention were perceived as relevant and acceptable. Participants noted that they could be more truthful with a digital counselor, however, it was not unanimously endorsed as a replacement for a human counselor. The C-Raven Satisfaction Scale showed that all participants rated their satisfaction at either a 4 (n=2) or a 5 (n=8) on a 5-point Likert scale and all endorsed using the C-Raven program again. CDIs show promise in extending access to care and improving health outcomes but their development necessarily requires integration from multiple disciplines including behavioral medicine and computer science. We developed a cross-platform compatible CDI led by a digital counselor that interacts in a motivational interviewing style and (1) uses evidence-based behavioral change methods, (2) is culturally adapted to people with HIV who use drugs, (3) has an engaging and interactive user interface, and (4) presents personalized content based on participants' ongoing responses to a series of menu-driven conversations. To advance the continued development of this and other CDIs, we recommend expanded testing, standardized measures to evaluate user experience, integration with clinician-delivered substance use treatment, and if effective, implementation into HIV clinical care.

Can digital health researchers make a difference during the pandemic? Results of the single-arm, chatbot-led Elena+: Care for COVID-19 interventional study.

The current paper details findings from Elena+: Care for COVID-19, an app developed to tackle the collateral damage of lockdowns and social distancing, by offering pandemic lifestyle coaching across seven health areas: anxiety, loneliness, mental resources, sleep, diet and nutrition, physical activity, and COVID-19 information. The Elena+ app functions as a single-arm interventional study, with participants recruited predominantly via social media. We used paired samples T-tests and within subjects ANOVA to examine changes in health outcome assessments and user experience evaluations over time. To investigate the mediating role of behavioral activation (i.e., users setting behavioral intentions and reporting actual behaviors) we use mixed-effect regression models. Free-text entries were analyzed qualitatively. Results show strong demand for publicly available lifestyle coaching during the pandemic, with total downloads (N = 7'135) and 55.8% of downloaders opening the app (n = 3,928) with 9.8% completing at least one subtopic (n = 698). Greatest areas of health vulnerability as assessed with screening measures were physical activity with 62% (n = 1,000) and anxiety with 46.5% (n = 760). The app was effective in the treatment of mental health; with a significant decrease in depression between first (14 days), second (28 days), and third (42 days) assessments: F2,38 = 7.01, p = 0.003, with a large effect size (η2G = 0.14), and anxiety between first and second assessments: t54 = 3.7, p = <0.001 with a medium effect size (Cohen d = 0.499). Those that followed the coaching program increased in net promoter score between the first and second assessment: t36 = 2.08, p = 0.045 with a small to medium effect size (Cohen d = 0.342). Mediation analyses showed that while increasing number of subtopics completed increased behavioral activation (i.e., match between behavioral intentions and self-reported actual behaviors), behavioral activation did not mediate the relationship to improvements in health outcome assessments. Findings show that: (i) there is public demand for chatbot led digital coaching, (ii) such tools can be effective in delivering treatment success, and (iii) they are highly valued by their long-term user base. As the current intervention was developed at rapid speed to meet the emergency pandemic context, the future looks bright for other public health focused chatbot-led digital health interventions.