User Experience Evaluation Research Articles

Evaluating the real-world usability of BCI control systems with augmented reality: a user study protocol.

Brain-computer interfaces (BCI) enable users to control devices through their brain activity. Motor imagery (MI), the neural activity resulting from an individual imagining performing a movement, is a common control paradigm. This study introduces a user-centric evaluation protocol for assessing the performance and user experience of an MI-based BCI control system utilizing augmented reality. Augmented reality is employed to enhance user interaction by displaying environment-aware actions, and guiding users on the necessary imagined movements for specific device commands. One of the major gaps in existing research is the lack of comprehensive evaluation methodologies, particularly in real-world conditions. To address this gap, our protocol combines quantitative and qualitative assessments across three phases. In the initial phase, the BCI prototype's technical robustness is validated. Subsequently, the second phase involves a performance assessment of the control system. The third phase introduces a comparative analysis between the prototype and an alternative approach, incorporating detailed user experience evaluations through questionnaires and comparisons with non-BCI control methods. Participants engage in various tasks, such as object sorting, picking and placing, and playing a board game using the BCI control system. The evaluation procedure is designed for versatility, intending applicability beyond the specific use case presented. Its adaptability enables easy customization to meet the specific user requirements of the investigated BCI control application. This user-centric evaluation protocol offers a comprehensive framework for iterative improvements to the BCI prototype, ensuring technical validation, performance assessment, and user experience evaluation in a systematic and user-focused manner.

Exploring the elderly user experience in using digital health interventions: a case study of the MySejahtera application

In recent years, there has been increased interest in using digital health tools to improve healthcare outcomes and encourage healthy lifestyles. However, older adults, who often face challenges with technology, may encounter difficulties when using these tools. This study focused on understanding how elderly individuals experience the MySejahtera app, a digital health tool designed to help manage health during a pandemic. The research involved 30 elderly users through focus groups, interviews, and user experience evaluations. The study identified key themes in the elderly user experience, such as the need for simplicity, efficiency in the app's interface, ease of learning, concerns about security, and emotional reactions. Recommendations were made to enhance the engagement of elderly users with digital health apps. While the MySejahtera app shows promise for older adults, the study highlights the importance of addressing specific design considerations and providing support to improve user satisfaction. Overall, the research offers valuable insights and recommendations for designing and implementing digital health applications that better meet the needs and preferences of elderly users.

Enhancing Driving Safety through User Experience Evaluation of the C-ITS Mobile Application: A Case Study of the DARS Traffic Plus App in a Driving Simulator Environment.

The paper evaluates the DARS Traffic Plus mobile application within a realistic driving simulator environment to assess its impact on driving safety and user experience, particularly focusing on the Cooperative Intelligent Transport Systems (C-ITS). The study is positioned within the broader context of integrating mobile technology in vehicular environments to enhance road safety by informing drivers about potential hazards in real time. A combination of experimental methods was employed, including a standardised user experience questionnaire (meCUE 2.0), measuring quantitative driving parameters and eye-tracking data within a driving simulator, and post-experiment interviews. The results indicate that the mobile application significantly improved drivers' safety perception, particularly when notifications about hazardous locations were received. Notifications displayed at the top of the mobile screen with auditory cues were deemed most effective. The study concludes that mobile applications like DARS Traffic Plus can play a crucial role in enhancing road safety by effectively communicating hazards to drivers, thereby potentially reducing road accidents and improving overall traffic safety. Screen viewing was kept below the safety threshold, affirming the app's efficacy in delivering crucial information without distraction. These findings support the integration of C-ITS functionalities into mobile applications as a means to augment older vehicle technologies and extend the safety benefits to a broader user base.

Evaluation of user experience and satisfaction with workplace-based assessments (WPBAs) in dental postgraduate programmes.

To explore the experience of and satisfaction with workplace-based assessment tools among dental postgraduate trainees. The cross-sectional study was conducted from March to October 2022 at the Aga Khan University Hospital, Karachi, and comprised all Operative Dentistry, Prosthodontics and Orthodontics postgraduate trainees. A questionnaire with both open-ended and closed-ended questions was used to record the experiences and satisfaction level of the dental residents who had earlier been subjected to workplace-based assessment. Data was analysed using SPSS 20, while thematic analysis was used for open-ended questions. Of the 20 Subjects, 15(75%) were females. Also, 11(55%) participants had received prior training for workplacebased assessment. There were 16(80%) residents who were satisfied with workplace-based assessment tools in the dental residency programme, 15(75%) agreed that the tools improved their clinical skills and helped them identify their weak areas, all the 20(100%) participants said the feedback given to them was constructive, 18(90%) reported that they were allowed to put in their views. However, 7(35%) participants reported that being observed adversely affected their performance. Dental residents generally gave positive feedback related to workplace-based assessment tools in a clinical setting.

Research on Perfume User Experience Evaluation Based on Human Machine Hybrid Generation Adversarial Network

Research on Perfume User Experience Evaluation Based on Human Machine Hybrid Generation Adversarial Network

Enhancing user experience in the digital service environment: A comprehensive study on the design and evaluation of internet-based healthcare products.

In the era of burgeoning digital technology, healthcare is a challenging transformative change towards virtual and digital platforms. Internet-based healthcare services are emerging as a popular trend within the medical area. User experience (UX) is paramount for the healthcare service, as it significantly influences experience satisfaction and fosters user viscosity. Gaining a profound understanding of users' demands and crafting services that align with their expectations is essential. Consequently, exploring an effective design approach for the digital healthcare service that prioritizes UX along with utilizing a comprehensive evaluation methodology to handle UX data, is of profound importance. This study introduces a design methodology for Internet-based healthcare products grounded in the UX and mental (UX-M) model. Aiming to refine the Internet-based healthcare product design by integrating insights from the experience data, it employs the Delphi-ANP and the fuzzy comprehensive evaluation to determine evaluation indexes and conduct experiential assessments. The UX evaluation results of existing schemes are compared with the proposed design scheme of the intelligent guidance and internet hospital. The findings indicate that the UX evaluation of Internet-based medical services with the proposed method outperforms the existing schemes. On the one hand, UX research of Internet-based healthcare products can significantly enhance service satisfaction for patients utilizing online medical treatments. On the other hand, the analysis of experience-based evaluation empowers designers to refine and improve UX design of Internet-based medical services. Such research endeavors are critical for enhancing the overall quality of service offerings and elevating user satisfaction in the digital healthcare landscape.

How do emotions evolve? The effect of emotions in the interactive scenes of a new product trial

As people pay more attention to users’ subjective experiences, emotion becomes more important for productions and designers. Emotion has been defined, classified, and evaluated through interaction process in different fields, but how emotions evolve during user interaction and affect the user experience bring a new research topic. In this paper, emotions are divided into instantaneous emotions (IE) and perceived emotions (PE) for research. As a result, different IE have different evolution rules (Curious trends to stay, Amazed trends to transform), and certain PE do have significant effects on user experience in different stages. Also, we find non-correspondence between IE and PE. The results provide designers with more details of users’ feedback and bring more help for emotion prediction and user experience evaluation.

An AI-powered approach to the semiotic reconstruction of narratives

This article presents a novel and highly interactive process to generate natural language narratives based on our ongoing work on semiotic relations, providing four criteria for composing new narratives from existing stories. The wide applicability of this semiotic reconstruction process is suggested by a reputed literary scholar’s deconstructive claim that new narratives can often be shown to be a tissue of previous narratives. Along, respectively, three semiotic axes – syntagmatic, paradigmatic, and meronymic – existing stories can yield new stories by the combination, imitation, or expansion of an iconic scene; lastly, a new story may emerge through reversal via an antithetic consideration, i.e., through the adoption of opposite values. Targeting casual users, we present a fully operational prototype with a simple and user-friendly interface that incorporates an AI agent, namely ChatGPT. The prototype, in a coauthor capacity, generates context-compatible sequences of events in storyboard format using backward-chaining abductive reasoning (employing Stable Diffusion to draw scene illustrations), conforming as much as possible to the user’s authorial instructions. The extensive repertoire of book and movie summaries available to the AI agent obviates the need to manually supply laborious and error-prone context specifications. A user study was conducted to evaluate user experience and satisfaction with the generated narratives. The preliminary findings suggest that our approach has the potential to enhance story quality while offering a positive user experience.

Effective Automation of End-to-End Testing for OTT Platforms

The rapid evolution of Over-the-Top (OTT) platforms has reshaped the digital media landscape, necessitating robust testing strategies to ensure seamless user experiences and service reliability. Effective automation of end-to-end testing for OTT platforms emerges as a crucial strategy to address the complexities associated with diverse content delivery, user interactions, and varying device compatibilities. This paper explores the challenges and best practices in automating end-to-end testing processes for OTT platforms, focusing on integrating various testing tools, frameworks, and methodologies to achieve comprehensive coverage and efficiency. OTT platforms deliver content directly over the internet, bypassing traditional broadcast and cable channels. This direct-to-consumer approach introduces unique challenges in testing, such as ensuring consistent performance across different devices, network conditions, and content types. Additionally, the dynamic nature of content libraries and frequent updates necessitate rigorous testing to identify and rectify potential issues proactively. Automation, with its capacity to simulate real-world usage scenarios, streamline repetitive tasks, and enhance testing accuracy, plays a pivotal role in addressing these challenges. The paper first outlines the primary challenges in end-to-end testing for OTT platforms. These challenges include the need for cross-platform compatibility, performance under varying network conditions, and the ability to handle diverse content formats and streaming protocols. The complexities are compounded by the requirement for real-time performance metrics and user experience evaluations, which demand a sophisticated testing infrastructure.

Https://gurukuljournal.com/virtual-classroom/

Abstract— This research paper presents the design, implementation, and evaluation of a virtual classroom system developed using Java. With the increasing demand for remote learning solutions, the project aims to provide an interactive and collaborative platform for students and instructors to engage in online education. The paper discusses the system architecture, features, implementation details, and user experience evaluation, highlighting the potential of Java-based virtual classrooms in modern education. IndexTerms – Web-Based Application, Event Creation, Event tracking, and School Academic Calendar, MERN Stack.

Learning by Doing in VR: A User-Centric Evaluation of Lathe Operation Training

This study presents the development and evaluation of an immersive virtual reality (VR) application designed for lathe operation training. The VR application, built using Unity for Oculus Rift headsets, aims to simulate a realistic lathe machining experience, allowing users to interact with the machine’s various controls and levers. The experimental analysis involved 20 s-year Mechanical Engineering students who performed machining tasks in the virtual environment. The usability and user experience of the application were assessed using the System Usability Scale (SUS) and a 12-item questionnaire. The SUS results yielded a high mean score of 96.25 (SD = 6.41), indicating excellent usability. The user experience evaluation also showed positive feedback, with high ratings for the sense of presence, realism, and usefulness for training purposes. However, some users reported minor physical discomforts such as dizziness. The study concludes that immersive VR is a valuable tool for enhancing training in lathe operations, offering an engaging and realistic experience that encourages active learning. Future work should focus on reducing physical discomfort and further improving the application’s realism and interactivity.

2F5-2 Relationship between daily user experience evaluation of products and overall satisfaction

2F5-2 Relationship between daily user experience evaluation of products and overall satisfaction

Characterizing UX Assessment in the Context of Immersive Experiences: A Systematic Mapping Study

An immersive experience is a multisensory experience across a journey or task that’s contextually relevant, enabled by a combination of interactions that create intuitive and emotional value for the user. As the technological landscape has evolved, immersive experiences have become increasingly integrated into our lives. The rise of immersive experiences comes with a focus on how to evaluate those experiences considering User Experience (UX). UX is a multifaceted construct, and its importance differs according to the type of experience. In this scenario, knowing how to evaluate UX is fundamental to understanding whether immersive experiences are pleasant. Despite some attempts to address the UX, a systematic approach to addressing UX in the immersive context still needs to be developed. This paper presents a Systematic Literature Mapping (SLM) to investigate how UX evaluations have been performed and the main UX dimensions that should be considered in immersive experiences, such as engagement, presence, and immersion. Our main result is a theoretical model that we proposed based on the UX definitions and relations from the literature. Our model can help study the relations between UX dimensions, establishing the primary UX dimensions regarding immersive experiences, and as a base for developing new UX evaluation techniques.

A Study on the Willingness of “Generation Z” Consumers to Use Online Virtual Try-On Shopping Services Based on the S-O-R Framework

Virtual try-on services, which significantly reduce return rates and enhance user shopping experiences, pose a crucial question: how can user willingness to use these services be increased? Additionally, “Generation Z” consumers, known for their unique traits and significant consumption potential, have been relatively understudied in this context. To address this theoretical gap, this study employs the Stimulus–Organism–Response (S-O-R) theory as its research framework, using online virtual shoe try-on services as a case study. Focusing on “Generation Z” consumers, this study utilizes literature review, user research, factor analysis, and linear regression to establish a user experience evaluation scale and behavior model. The user experience evaluation scale identifies positive elements such as convenience, price value, visual information acquisition, emotional value, and social interaction. The negative elements include technical limitations, personalized service deficiencies, and uncertainty. In the behavior model, all elements influence user attitudes. Notably, only the negative elements directly affect users’ willingness to use the service, except social interaction among the positive elements, which positively impacts usage intention. This study also reveals that “Generation Z” consumers have novel requirements for social interaction. This research effectively addresses existing theoretical gaps and provides a foundational theory for the development of related services and technologies.

Point-of-need mastitis pathogen biosensing in bovine milk: From academic sample preparation novelty to industry prototype field testing

Bovine mastitis is an inflammation of the mammary gland, and it is the most common infectious disease in dairy cattle. Mastitis reduces milk yield and quality, costing dairy farmers millions of dollars each year. The aim of this study was to develop a point-of-need test for identifying mastitis pathogens that is field portable, cost-effective and can be used with minimal training. Using a proprietary polymer-based milk sample preparation method to rapidly extract pathogen DNA in milk samples, we demonstrated quantitative Polymerase Chain Reaction (qPCR) assays for six common bovine bacterial mastitis pathogens: Staphylococcus aureus, Streptococcus agalactiae, Streptococcus dysgalactiae, Streptococcus uberis, Mycoplasma bovis and Escherichia coli. We also implemented this sample preparation method on a prototype point-of-need system in a proof-of-concept field trial to evaluate user experience. Importantly, the protype system enabled a sample-to-result turnaround time of within 70 min to quantitatively detect all six target pathogens. The key advantage of our point-of-need prototype system is being culture-independent yet providing automated milk sample preparation for molecular identification of key mastitis pathogens by non-expert users. Our point-of-need prototype system showed a good correlation to laboratory-based qPCR for target pathogen detection outcomes, thus potentially removing the need for milk samples to be transported off-site for laboratory testing. Above all, we successfully achieved our objective of developing a point-of-need biosensor technology for mastitis and increased its readiness level with industry partners towards technology commercialization.

A Question Answering Chatbot for Gastric Cancer Patients After Curative Gastrectomy: Development and Evaluation of User Experience and Performance.

Postoperative gastric cancer patients have many questions about managing their daily lives with various symptoms and discomfort. This study aimed to develop a knowledge-based question answering (QA) chatbot for their self-management and to evaluate the user experience and performance of the chatbot. To support the chatbot's natural language processing, we analyzed QA texts from an online self-help group, clinical guidelines, and refined frequently asked questions related to gastric cancer. We developed a named entity classification with seven superconcepts, 4544 subconcepts, and 1415 synonyms. We also developed a knowledge base by linking the users' classified question intents with the experts' answers and knowledge resources, including 677 question intents and scripts with standard QA pairs and similar question phrases. A chatbot called "GastricFAQ" was built, reflecting the question topics of the named entity classification and QA pairs of the knowledge base. User experience evaluation (N = 56) revealed the highest mean score for usefulness (4.41/5.00), with all other items rated 4.00 or higher, except desirability (3.85/5.00). The chatbot's accuracy, precision, recall, and F score ratings were 85.2%, 87.6%, 96.8%, and 92.0%, respectively, with immediate answers. GastricFAQ could be provided as one option to obtain immediate information with relatively high accuracy for postoperative gastric cancer patients.

Active convolutional neural networks sign language (ActiveCNN-SL) framework: a paradigm shift in deaf-mute communication

Real-time speech-to-text and text-to-speech technologies have significantly influenced the accessibility of communication for individuals who are deaf or mute. This research aims to assess the efficacy of these technologies in facilitating communication between deaf or mute individuals and those who are neither deaf nor mute. A mixed-method approach will incorporate qualitative and quantitative data collection and analysis techniques. The study will involve participants from deaf or mute and non-deaf or non-mute communities. The research will scrutinize the precision and efficiency of communication using these technologies and evaluate user experience and satisfaction. Furthermore, the study intends to pinpoint potential obstacles and limitations of these technologies and offer suggestions for enhancing their effectiveness in fostering inclusivity. The study proposes an active learning framework for sign language gesture recognition, termed Active Convolutional Neural Networks—Sign Language (ActiveCNN-SL). ActiveCNN-SL aims to minimize the labeled data required for training and augment the accuracy of sign language gesture recognition through iterative human feedback. This proposed framework holds the potential to enhance communication accessibility for deaf and mute individuals and encourage inclusivity across various environments. The proposed framework is trained using two primary datasets: (i) the Sign Language Gesture Images Dataset and (ii) the American Sign Language Letters (ASL)—v1. The framework employs Resnet50 and YoloV.8 to train the datasets. It has demonstrated high performance in terms of precision and accuracy. The ResNet model achieved a remarkable accuracy rate of 99.98% during training, and it also exhibited a validation accuracy of 100%, surpassing the baseline CNN and RNN models. The YOLOv8 model outperformed previous methods on the ASL alphabet dataset, achieving an overall mean average accuracy for all classes of 97.8%.

Analysis UX Design e-Commerce "Key Kaos" with Lean UX

This research explores the evaluation of User Experience (UX) design at the Key Kaos online store using the Lean UX approach. Key Kaos online store required user-centered design to increase customer satisfaction and drive engagement. This study integrates Lean UX principles to simplify the design process, encourage collaboration, and rapid iteration.The research methodology includes a combination of user feedback, usability testing, and iterative prototyping, which aligns with the Lean UX framework. Through this iterative process, this research aims to identify key weak points and areas of improvementin the existing UX design of the Key Kaos online store. These findings are expected to provide valuable insights in increasing user satisfaction, improving usability, and optimizing the overall user journey in the market.Additionally, this research explores the impact of Lean UX on cross-functional collaboration, aiming to identify how Lean UX facilitates communication and collaboration between designers, developers and other stakeholders. By assessing the effectiveness ofLean UX in the context of Key Kaos, this research contributes to a broader understanding of its application in the market environment.The expected outcomes of this research include actionable recommendations to improve the UX design of Key Kaos online stores, ultimately contributing to market success by creating a more user-friendly and efficient platform. Additionally, research insightsregarding the application of Lean UX principles can inform future efforts in marketplace UX design, offering a valuable reference for designers and researchers looking to optimize user experiences in online marketplaces.

Evaluating User Experience and DNA Yield from Self-Collection Devices.

The COVID-19 pandemic emphasized an urgent need for devices used in the self-collection of biospecimens in an evolving patient care system. The mailing of biospecimen self-collection kits to patients, with samples returned via mail, provides a more convenient testing regimen, but could also impart patient sampling variabilities. User compliance with device directions is central to downstream testing of collected biospecimens and clear instructions are central to this goal. Here, we performed an evaluation of 10 oral DNA collection devices involving either swab or saliva self-collection and analyzed ease of use and comfort level with a device, as well as DNA recovery quantity/quality and sample stability. We show that while these DNA quality/quantity metrics are comparable between devices, users prefer direct saliva collection over swab-based devices. This information is useful in guiding future experiments including their use in human RNA, microbial, or viral sample collection/recovery and their use in clinical testing.

IDENTIFYING AND FIXING UX-FRICTION EMPLOYEE DEVELOPMENT WEBSITE WITH MIXED APPROACH HEART FRAMEWORK AND USABILITY TESTING

The company website has an important role in the business processes that occur in the company, one of which is managing employee productivity. PT.XYZ utilizes the Kaizen website to improve the quality of its employees. Assessment of user experience while operating the website can be a form of evaluation between expectations and actual conditions related to website feasibility. In this study, the researcher intends to find out what indicators are obstacles in the operation of a web-based application called the Kaizen website at PT.XYZ so that there is inefficiency in its use. This research involved 54 respondents who were employees of PT.XYZ. The combination of HEART and Usability Testing methods is used to identify obstacles experienced by users and evaluate user experience when using the Kaizen website. The results of the analysis are then used to improve website performance in order to create a good user experience. The results showed that the HEART score increased by an average of 9% on each variable. Usability testing assessment also shows that the Kaizen website reaches the acceptable category with a score of 90.5 after improvements are made.