Usability Scale Questionnaire Research Articles

Empowering Active and Healthy Ageing: Integrating IoT and Wearable Technologies for Personalised Interventions

Social isolation and loneliness greatly contribute to negative health consequences in older adults. Technological solutions can be an asset in promoting social connections and healthy behaviours. This paper presents an innovative structure for an Internet of Things (IoT) platform specifically tailored for older persons. The framework utilises a supervised learning algorithm to classify users into four identified profiles to facilitate the adoption and engagement of technology. The platform incorporates wearables, such as socks and smart bands, to track physical activity, and a messaging module to encourage social interaction. The platform processes the acquired data to quantify steps and deliver tailored interventions remotely to the older adults through the AGAPE Assistant, the mHealth solution of the platform. Furthermore, the AGAPE Assistant has a user interface design for older adults, with a focus on their specific needs. Additionally, improving digital literacy among older adults is crucial for maximizing the long-term compliance and benefits of such technological solutions. On the other hand, AGAPE Monitor is a web application used by formal caregivers to configure the tailored interventions. The platform’s usability was assessed using different usability scale questionnaires, which revealed a mild level of user satisfaction and acceptance. The proposed framework is currently being deployed on more than 112 older adults across three countries: Italy, Romania, and Portugal. The proposed framework provides a holistic solution to encourage active ageing by adopting technology, implementing hybrid interventions, and promoting social interactions.

User-centered design and spatially-distributed sequential electrical stimulation in cycling for individuals with paraplegia

BackgroundIn this work, we share the enhancements made in our system to take part in the CYBATHLON 2020 Global Edition Functional Electrical Stimulation (FES) Bike Race. Among the main improvements, firstly an overhaul, an overhaul of the system and user interface developed with User-centered design principles with remote access to enable telerehabilitation. Secondly, the implementation and experimental comparison between the traditional single electrode stimulation (SES) and spatially distributed sequential stimulation (SDSS) applied for FES Cycling.MethodsWe report on the main aspects of the developed system. To evaluate the user perception of the system, we applied a System Usability Scale (SUS) questionnaire. In comparing SDSS and SES, we collected data from one subject in four sessions, each simulating one race in the CYBATHLON format.ResultsUser perception measured with SUS indicates a positive outcome in the developed system. The SDSS trials were superior in absolute and average values to SES regarding total distance covered and velocity. We successfully competed in the CYBATHLON 2020 Global Edition, finishing in 6th position in the FES Bike Race category.ConclusionsThe CYBATHLON format induced us to put the end-user in the center of our system design principle, which was well perceived. However, further improvements are required if the intention is to progress to a commercial product. FES Cycling performance in SDSS trials was superior when compared to SES trials, indicating that this technique may enable faster and possibly longer FES cycling sessions for individuals with paraplegia. More extensive studies are required to assess these aspects.

The evaluation of a digital solution to improve the implementation of NCCN guidelines.

e13542 Background: NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines) are the most commonly used guidelines in the US and globally, and aim to improve and facilitate quality, effective, efficient and accessible cancer care. The guidelines are constantly updated, as such, the number, scope, length and detail have continuously increased, which has placed challenges for clinicians to remain up to date. Solutions are highly needed to improve the usability and implementation of the NCCN Guidelines. The NAVIFY Guidelines application was developed to provide digitised NCCN Guidelines, to facilitate navigation, and to enable the real-time capture of clinical decisions, including reasons for deviation from guidelines. We assessed the usability and performance of the NAVIFY Guidelines application. Methods: A 60-minute remote, semi-structured interview and task-based user test was undertaken with 10 medical oncologists from diverse geographies (6 US, 2 UK, 2 Germany). No participants had prior experience with the digital solution. Participants tested the experience of using the NAVIFY Guidelines application on their computer by completing tasks for one of 3 tumor types (breast, lung or AML). Screen and audio activities were recorded via Zoom. At the end of the test, participants were asked to rate the performance of NAVIFY Guidelines and complete statements on a System Usability Scale (SUS) questionnaire. Results: NAVIFY Guidelines SUS score was on average 82.3 (80.8 – 84.0), classified as A score (i.e., excellent; NPS [net promoter score]: promoter level). Success ratings across all 6 tasks was 86%, as shown in Table1. The overall impression from participants was largely positive. "Ease of use" was the most salient theme followed by “logical structure” and the “value of being integrated with patient data”. Participants identified that the key value of NAVIFY Guidelines was improving communication between the care team and improving the documentation of guideline adherence. 3 participants saw the potential in teaching settings. Conclusions: NAVIFY Guidelines has been developed to support clinicians to apply the latest clinical recommendations, improve the documentation of decisions and to facilitate better communication between specialists, and with patients. Preliminary research has shown that NAVIFY Guidelines has excellent usability (score A) & high NPS. Overall NAVIFY Guidelines was perceived as easy to use, valuable to improve communication between care teams and helpful at easing documentation burdens.

Usability and Emotions of Mental Health Assessment Tools: Comparing Mobile App and Paper-and-Pencil Modalities

Users’ experiences in mental health assessment are multifaceted, including their emotional experiences. Yet, studies of mobile apps for psychiatric assessment have centered on diagnostic accuracy and perceived usability, with little consideration of the impact of user emotional experiences. In this study, we focused on users’ perceived usability and emotions and compared the user experience of a paper-and-pencil and an app-based collection of mental health screening questionnaires: EarlyDetect. The System Usability Scale (SUS) and modality-directed emotion questionnaires were administered using paper-and-pencil or iPad. Modality was assigned pseudo-randomly on patients’ first visit at a referral-based mental health clinic. We found that patients assigned to the iPad app reported a significantly higher SUS score than patients assigned to paper-and-pencil, qualified by a modality-by-gender interaction where modality effects were significant for men but not for women. Moreover, enjoyment was positively linked to perceived usability, whereas boredom, frustration, and anxiety were negatively linked to usability. Our findings illustrate the added value of studying user experience applied to psychiatric assessments, where both emotions and gender-specific user experience should be taken into consideration. We further discuss the implications for psychiatric assessments via app versus traditional data collection.

A Simulated Graphical Interface for Integrating Patient-Generated Health Data From Smartwatches With Electronic Health Records: Usability Study.

BackgroundWearable technology, such as smartwatches, can capture valuable patient-generated data and help inform patient care. Electronic health records provide logical and practical platforms for including such data, but it is necessary to evaluate the way the data are presented and visualized.ObjectiveThe aim of this study is to evaluate a graphical interface that displays patients’ health data from smartwatches, mimicking the integration within the environment of electronic health records.MethodsA total of 12 health care professionals evaluated a simulated interface using a usability scale questionnaire, testing the clarity of the interface, colors, usefulness of information, navigation, and readability of text.ResultsThe interface was positively received, with 14 out of the 16 questions generating a score of 5 or greater among at least 75% of participants (9/12). On an 8-point Likert scale, the highest rated features of the interface were quick turnaround times (mean score 7.1), readability of the text (mean score 6.8), and use of terminology/abbreviations (mean score 6.75).ConclusionsCollaborating with health care professionals to develop and refine a graphical interface for visualizing patients’ health data from smartwatches revealed that the key elements of the interface were acceptable. The implementation of such data from smartwatches and other mobile devices within electronic health records should consider the opinions of key stakeholders as the development of this platform progresses.

Evaluation on Augmented-Reality-Based Interactive Book Using System Usability Scale and User Experience Questionnaire

Nowadays, the implementation of information and communication technology in education is important. In harmony with technological developments, the term mobile learning comes to represent learning that utilizes mobile communication devices. The implementation of augmented reality in education provides a new learning model in the form of a combination of technology-based conventional learning media. This research aimed to evaluate augmented reality-based books as media for learning Cirebon mask dance. As many as 15 respondents were involved in the evaluation process, including testing the usability using the System Usability Scale (SUS) and User Experience Questionnaire (UEQ). The evaluation using SUS found a value of 77.67, meaning that the Acceptability Ranges category was “Acceptable”; the Grade Scale category was “C”; and the Adjective Rating category was “Excellent”, while that using UEQ found that each category, namely Attractiveness, Perspicuity, Efficiency, Dependability, Stimulation, Dependability, and Novelty got a value greater than the impression value (0.8), namely 2.122; 2.117; 1.983; 1.750; 1.950 and 1,867, respectively. Overall, all of the evaluation results show that augmented reality-based books are acceptable to be used as media for learning Cirebon mask dance.

OTP-PRL: an app for occupational risk prevention in policing activities

BackgroundThe rapid progress in information and communication technologies has promoted the development of smartphone applications (apps) for a wide variety of purposes including workplace safety. However, no specific apps for occupational risk prevention in the police domain seemingly exist as yet. In this work, an app running under the iOS and Android operating systems was developed to help police officers become acquainted with policing-related occupational risks and to prevent their damaging consequences.ResultsThe proposed app, which uses an iterative user-centered design to avoid occupational risks in policing activities, was assessed for performance by a group of users and experts through a System Usability Scale (SUS) questionnaire. The mean overall score for the questionnaire was 82.3. The app has backend support to facilitate continual improvement through contributions from users and administrators. A field test revealed increased awareness of policing occupational risks after using the app in many users.ConclusionsA novel product that covers the needs of ORP requirements of police officers has been developed.

Scaled Pearson’s Correlation Coefficient for Evaluating Text Similarity Measures

Despite the ever-increasing interest in the field of text similarity methods, the development of adequate text similarity methods is lagging. Some methods are decent in entailment while others are reasonable to the degree to which two texts are similar. Very often, these methods are compared using Pearson’s correlation; however, Pearson’s correlation is bound to outliers that could affect the final correlation coefficient figure. As a result, the Pearson correlation is inadequate to find which text similarity method is better in situations where data items are very similar or are unrelated. This paper borrows the scaled Pearson correlation from the finance domain and builds a metric that can evaluate the performance of similarity methods over cross-sectional datasets. Results showed that the new metric is fine-grained with the benchmark dataset scores range as a promising alternative to Pearson’s correlation. Moreover, extrinsic results from the application of the System Usability Scale (SUS) questionnaire on the scaled Pearson correlation revealed that the proposed metric is attaining attention from scholars which implicate its usage in the academia.

Falls Sensei: a serious 3D exploration game to enable the detection of extrinsic home fall hazards for older adults

BackgroundFalls are the main cause of death and injury for older adults in the UK. Many of these falls occur within the home as a result of extrinsic falls risk factors such as poor lighting, loose/uneven flooring, and clutter. Falls education plays an important role in self-management education about extrinsic hazards and is typically delivered via information leaflets, falls apps, and educational booklets. Serious games have the potential of delivering an engaging and informative alternative to traditional methods but almost exclusively, these are currently delivered as exergaming applications that focus solely on intrinsic falls risk factors. This study presents ‘Falls Sensei’ a first-person 3D exploration game that aims to educate older adults about extrinsic falls risk factors within the home environment. After presenting Falls Sensei, game usability and older adults’ perceptions and attitudes towards using the game in practice are explored.MethodsThis study involved 15 community dwelling older adults. After playing the Falls Sensei game, participants completed a Systems Usability Scale (SUS) questionnaire and post task interview, and follow-up interviews three weeks later. Inductive and deductive thematic template analysis, informed by the Unified Theory of Acceptance and Use of Technology model, was used to analyse the think-aloud, post-task and follow-up interview transcripts. Descriptive statistical analysis and one-sampled t-tests were used to analyse log-file data and SUS responses.ResultsThree high-level themes emerged from the analysis of transcriptions: Performance Expectancy; Effort Expectancy; Social Influence. The SUS score was 77.5/100 which indicates ‘Good’ levels of usability. Interestingly, reported usability of the game increased with participant age. Participants were positive about the usability of the game (p < = 0.05 for 9/10 items). The most memorable fall hazards were those most commonly encountered in the game or those most challenging to participants.ConclusionsThe results support the use of serious games as an engaging tool for educating older adults about extrinsic falls risk factors. Awareness of home hazard detection was raised by the game, and some older adults became more aware for the need to adapt their own homes after gameplay. Further research would be needed to draw comparisons with established interventions.

Timeline representation of clinical data: usability and added value for pharmacovigilance

BackgroundPharmacovigilance consists in monitoring and preventing the occurrence of adverse drug reactions (ADR). This activity requires the collection and analysis of data from the patient record or any other sources to find clues of a causality link between the drug and the ADR. This can be time-consuming because often patient data are heterogeneous and scattered in several files. To facilitate this task, we developed a timeline prototype to gather and classify patient data according to their chronology. Here, we evaluated its usability and quantified its contribution to routine pharmacovigilance using real ADR cases.MethodsThe timeline prototype was assessed using the biomedical data warehouse eHOP (from entrepôt de données biomédicales de l’HOPital) of the Rennes University Hospital Centre. First, the prototype usability was tested by six experts of the Regional Pharmacovigilance Centre of Rennes. Their experience was assessed with the MORAE software and a System and Usability Scale (SUS) questionnaire. Then, to quantify the timeline contribution to pharmacovigilance routine practice, three of them were asked to investigate possible ADR cases with the “Usual method” (analysis of electronic health record data with the DxCare software) or the “Timeline method”. The time to complete the task and the data quality in their reports (using the vigiGrade Completeness score) were recorded and compared between methods.ResultsAll participants completed their tasks. The usability could be considered almost excellent with an average SUS score of 82.5/100. The time to complete the assessment was comparable between methods (P = 0.38) as well as the average vigiGrade Completeness of the data collected with the two methods (P = 0.49).ConclusionsThe results showed a good general level of usability for the timeline prototype. Conversely, no difference in terms of the time spent on each ADR case and data quality was found compared with the usual method. However, this absence of difference between the timeline and the usual tools that have been in use for several years suggests a potential use in pharmacovigilance especially because the testers asked to continue using the timeline after the evaluation.

Order effect when comparing usability of two telehealth remote monitoring technology solutions

An experienced healthcare device evaluator collaborated with a human factors scientist to develop and conduct a usability test of two different telehealth remote monitoring solutions as part of a larger technology evaluation. Twenty-one participants used each of the telehealth platforms to capture and transmit data. Typically users of telehealth platforms capture and send their own data, however, we asked our participants to place a bag of sand on wirelessly connected scales to capture weight information and to place the wireless pulse oximeters on the healthcare device evaluator’s index finger to get blood oxygen saturation measurements. This protocol enabled participants to go through the processes required to capture and share representative data without providing any of their own personal health data. After using each solution to complete the same series of three tasks, participants filled out the System Usability Scale (SUS) questionnaire. We intended to compare SUS scores within-subjects, but discovered that the SUS scores for the second solution were not independent. In particular, the participants who used the system that was harder to use second produced SUS scores for that system that were significantly lower than the participants who rated that system first. We conclude that it may not be appropriate to use a within-subjects design if experimenters plan to compare the SUS scores for different systems.

The Design and Evaluation of an Affective Tutoring System

Emotion recognition techniques in an affective tutoring system enhance the affective interface of an intelligent tutoring system. This study attempts to improve interest in learning by recognizing the emotional states of students during their learning processes and providing adequate feedback. Both the emotional recognition system and the tutoring strategy module are developed. The digital arts learning content module is then designed, along with the emotional feedback mechanism via the HCI design of interactive avatars. Next, the entire system is integrated and assessed via a two-stage evaluation. Analysis results indicate that the proposed affective interaction design can enhance learner motivation and interest, thus improving their learning performance. System satisfaction and student performance are evaluated first by prototype evaluation, in which experts combine a system usability scale (SUS) questionnaire and heuristic evaluation. Triangulation evaluation is then performed, including qualitative and quantitative approaches involving observations, questionnaires, and interviews. The following questions are raised using the above methods: Is the usability of ATS effective?; Are ATS users satisfied with the learning system?; Is the interactivity of ATS attractive to users?; Does ATS increase the learning motivation of digital art students?; and Does ATS vary in its impact on self-conscious learning achievement for users from different domains? Evaluation results demonstrate a high satisfaction among users with respect to system usability. Furthermore, in addition to its attractiveness to users, ATS increases learning motivation and self-conscious learning achievement.