Usability Testing Research Articles

ARCHIGENIUS - VR MAZE SOLVING USING A* ALGORIHTM

Game is one of the entertainment media that is rapidly growing. On this day, playing games is one way to spare time for refreshing. One of the popular games this year is an action game such as First-Person Shooter (FPS). Besides, the FPS game genre has developed rapidly to make it more interesting for the player. Non-player Character (NPC) has an essential function in artificially and automatically to make a game more immersive and consisting of various kinds. Also, from observation, adding challenges when playing, namely enemy characters, can make a player more enjoyable. Besides, one of modern technology to develop innovations for supporting a game is called virtual reality. So, in this research, the FPS game is built by adding NPC elements with virtual reality technology. Also, the gameplay in this game is a survival maze. Based on the evaluation, this game application can display objects in a virtual reality smoothly, and all feature in this game can display and run effectively. Besides, the respondents also give good feedback for user testing which has average score 4.66 from several users. So, the conclusion from this research is this game can make the FPS game more interesting for users by adding NPC, 3D element, and virtual reality. Keywords - First-person Game, Virtual Reality, Survival Maze, Non-Player Character, virtual maze environment.

Unity-based Chinese Digital Education Game: Journey to Cultural Relics of Ancient China

This project is the study of a digital educational game primarily designed to aid foreigners in the process of learning Chinese characters and Chinese culture. The game was created to be three-dimensional, incorporating semi-realistic traditional Chinese art-style illustrations. In addition to the incorporated card memory game, designed to help memorize Chinese characters, there are also elements used to introduce the culture and history of China's four great inventions, assimilating them into four task-setting mini-games. Through this method, players can deepen their impression of Chinese culture by playing mini-games. The game was developed based on the Unity development platform, utilizing Maya for modeling, Substance Painter for textures, employing PS and Proceate for material painting, and interface design. From the interview results of the user test, the game has achieved the purpose of helping players learn Chinese characters, understand Chinese culture, and increase their interest in China.

CHATBOT USING LARGE LANGUAGE MODEL

The concept of Natural Language Processing has seen a remarkable advancement in the recent years. This remarkable advancement was particularly with the development of Large Language Models (LLM). Large Language Models are used to develop a human like conversations. This LLM is a part of Natural Language Processing which focuses on enabling computers to understand, interpret, and generate human language. The existing system of chatbots does not generate human like responses. The proposed system of chatbots uses the power of Large Language Models to generate more human like responses, providing the conversation in a natural way. By genereating human like respones, it will be in a natural way for the user. To enhance user experience, the chatbot uses a dynamic learning mechanism, by which it continuously adapt to user preferences and evolving conversational patterns. This system uses feedbacks from the users to refine its responses everytime.Moreover, the chatbot is designed with a multi-turn conversational context awareness, allowing it to maintain coherence and relevance throughout extended dialogues.The effectiveness of the proposed chatbot is evaluated through user testing, comparing its performance against traditional rule-based chatbots and existing conversational agents. This report explains about the usage of Large Language Models in the design and implementation of conversational chatbots. The outcomes of this research contribute to the advancement of intelligent chatbot systems, demonstrating the potential of large language models to significantly enhance conversational AI applications.

A framework for program wide curriculum transformation

Designing and delivering higher education programs in a global climate of constant change, technological advances and uncertain futures leads to the need for curriculum transformation practices that are innovative and responsive. This paper describes a university-wide approach to developing a framework for program level transformation that is strengths-based, data-informed and design-led. A strengths based approach builds on good practice, creating a space that is positive and forward looking. Data informed practice and the inclusion of data wranglers on the project allowed for conversations about the known, unknowns and desirable directions to take place and inform directions. Design-led practices introduced design thinking principles such a building empathy and co-design with students, alumni and industry. The emergent framework has three key stages: vision, design and build. The vision stage focuses on the program team, its students, industry and desired direction for transformation. The design stage focuses on defining challenges, ideating, co-designing and creating a plan for development. The build stage uses a rapid prototyping and iterative approach to development that incorporates user testing early in the stage. The project has delivered a framework for program level transformation and innovation and has shown that a strengths-based approach that is data informed and engages with students as codesigners has the capacity to unite teams, inform program visions and allow for innovative practices to emerge. Taking a learner experience approach to design also highlighted the value in engaging students and industry in curriculum design from the start of the process rather than simply as end users.

ProQuiz

ProQuiz is an app designed for educational purposes, specifically to help users prepare for multiple-choice exams, especially in the field of computer science. The app aims to make studying enjoyable by incorporating elements of gamification. The primary objective of ProQuiz is to assist users in getting ready for subjective tests used in admissions and recruitment processes, with a focus on the computer science domain. The app is organized into three main modules: computer science, reasoning, and analytical skills. Each of these modules covers various sub-categories to address different subjects. During the quiz, users have access to helpful features such as hints, the ability to skip questions, and a pause/life-lines option. These features can be used only once per user. The app also provides feedback on progress throughout the quiz, and upon completion, it displays the user's results. The overall aim is to provide users with an engaging platform for practicing and enhancing their knowledge in a variety of subjects related to computer science. This research paper examines the development and effectiveness of ProQuiz. Through user testing and analysis, we evaluate its usability and engagement. Additionally, we propose enhancements, including personalized learning paths and social features. This paper contributes insights into innovative educational technology and suggests avenues for future improvement of ProQuiz.

Pagelaran Pottery Village: Culture-Education Resin-Concrete Souvenir in Industrial Tourism

This implementation aims to develop resin-concrete souvenirs as a superior product for the pottery industry tourism in Pagelaran Village, with the aim of promoting cultural education and enhancing the visitor experience. Pagelaran Village, renowned for its heritage in pottery production and traditional cultural practices, faces challenges in preserving its cultural identity and attracting tourists amid changing consumer preferences. Consequently, this activity explores the concept of resin-concrete souvenirs as a method to convey the cultural identity of Pagelaran Village pottery according to tourist preferences. Implementation of this activity employed a mixed approach, combining qualitative interviews with local artisans and quantitative surveys of tourists, to gather insights into the village’s cultural heritage and visitor preferences. The resin-concrete souvenir prototype in the form of merchandise was designed by amalgamating traditional pottery motifs and cultural elements unique Pagelaran Village. Subsequently, it underwent user tests and feedback sessions to ensure their aesthetic appeal, durability, and ability to evoke cultural narratives. The resin-concrete souvenir prototype has significant potential as a superior product in industrial tourism. The integration of traditional pottery with modern materials enables the creation of products that are both visually appealing and culturally relevant. Additionally, tourist responses show high interest in souvenirs characterized by authentic and narrative cultural experiences. This implementation contributes to the development of souvenirs in the context of industrial tourism, emphasizing the significance of education and preservation of cultural heritage. Besides bolstering the economy of Pagelaran Village, resin-concrete souvenirs can educate visitors about the village’s cultural heritage, thereby encouraging sustainable tourism practices. Keywords: resin, concrete, souvenir, pottery, pagelaran

Development and Usability Evaluation of VulcanH, a CMMS Prototype for Preventive and Predictive Maintenance of Mobile Mining Equipment

This paper details the design, development, and evaluation of VulcanH, a computerized maintenance management system (CMMS) specialized in preventive maintenance (PM) and predictive maintenance (PdM) management for underground mobile mining equipment. Further, it aims to expand knowledge on trust in automation (TiA) for PdM as well as contribute to the literature on explainability requirements of a PdM-capable artificial intelligence (AI). This study adopted an empirical approach through the execution of user tests with nine maintenance experts from five East-Canadian mines and implemented the User Experience Questionnaire Plus (UEQ+) and the Reliance Intentions Scale (RIS) to evaluate usability and TiA, respectively. It was found that the usability and efficiency of VulcanH were satisfactory for expert users and encouraged the gradual transition from PM to PdM practices. Quantitative and qualitative results documented participants’ willingness to rely on PdM predictions as long as suitable explanations are provided. Graphical explanations covering the full spectrum of the derived data were preferred. Due to the prototypical nature of VulcanH, certain relevant aspects of maintenance planning were not considered. Researchers are encouraged to include these notions in the evaluation of future CMMS proposals. This paper suggests a harmonious integration of both preventive and predictive maintenance practices in the mining industry. It may also guide future research in PdM to select an analytical algorithm capable of supplying adequate and causal justifications for informed decision making. This study fulfills an identified need to adopt a user-centered approach in the development of CMMSs in the mining industry. Hence, both researchers and industry stakeholders may benefit from the findings.

Enhancing Museum Experience through Augmented Reality: The Case of the Indonesian Postal Museum

This paper presents the development and implementation of Mussia AR, an interactive augmented reality (AR) application designed for the Indonesian Postal Museum. Aimed at enhancing the educational and engagement aspects of museum visits, this project addresses the need for innovative approaches in museum experiences. We followed an Extreme Programming methodology for the development, ensuring a user-centric and iterative approach. The application overlays digital information onto physical exhibits, providing visitors with an immersive and informative experience. Our development process included comprehensive user needs analysis, application design, implementation, and extensive testing for both functionality and user experience. The results from user testing indicate a significant improvement in visitor engagement and satisfaction. The application not only succeeded in providing an enhanced learning experience but also demonstrated the potential of AR technology in cultural and educational settings. Future recommendations include expanding the content, introducing multilingual support, and extending the application's compatibility to various platforms. Mussia AR stands as a testament to the effective use of AR in enriching educational experiences in museums.

Enhancing statistical chart accessibility for people with low vision: insights from a user test

AbstractA remote user test was performed with two versions (one accessible and one non-accessible) of three types of web-based charts (horizontal bar chart, vertical stacked bar chart, and line chart). The objectives of the test were: (a) to validate a set of heuristic indicators for the evaluation of the accessibility of statistical charts presented in a previous work (Fariñas Falcón et al. in Mediocentro Electrónica 21(1):65–68, 2017); (b) to identify new barriers and preferences for users with low vision in the access and use of this content not previously contemplated. 12 users were tested, with a variety of conditions associated with low vision: low visual acuity (6 users), reduced central vision (2 users), reduced peripheral vision (2 users), blurry vision (1 user), sensitivity to light (3 users), Nystagmus (2 users) and color vision deficiency (CVD) (4 users). From a quantitative standpoint, accessible versions of charts were more efficient, effective, and satisfactory. From a qualitative point of view, results verify the relevance of heuristics H2, Legend; H3, Axes; H6, Data source (as data table); H10, Safe colors; H11, Contrast; H12, Legibility; H13, Image quality; H14, Resize; H16, Focus visible; H17, Independent navigation; related to the proposed tasks. As new observations, tooltips were highly valued by all users, but their implementation must be improved to avoid covering up significant parts of the charts when displayed. The data table has also been frequently used by all users, especially in the non-accessible versions, allowing them to carry out tasks more efficiently. The position and size of the legend can be a significant barrier if it is too small or appears in an unusual position. Finally, despite the limitations related to color perception, some users prefer color graphics to black and white, so, to target all profiles, it is necessary to redundantly encode categories with colors and patterns as well.

Developing Podcast-Based Learning Media for English Education Among Deaf Students

This study addresses the educational challenges faced by deaf students in Indonesia, focusing on developing and implementing a specialised Android-based application, "PODclusive," to facilitate English language learning. The research aimed to create an interactive learning tool that incorporates sign language and textual translations in both English and Indonesian, tailored to the needs of deaf learners. The methodology was a mixed-method approach, combining qualitative and quantitative analyses. The research and development (R&D) framework guided the application's design, development, and evaluation. Participants included a purposive sample of 30 deaf students from SLB N Pringsewu, ensuring a representation across different learning outcomes. The findings revealed that the PODclusive application significantly enhanced the learning experience for deaf students. Students demonstrated improved comprehension of English language concepts, facilitated by the application’s use of sign language video content and offline accessibility. The application was evaluated through expert reviews and user testing, receiving high validity scores for content, design, and educational impact. The implications of this study underscore the effectiveness of tailored educational technologies in addressing specific learning needs. The PODclusive application exemplifies how digital tools can enhance accessibility and engagement for deaf students, suggesting that further exploring similar technologies could be beneficial. This research contributes to the broader discourse on inclusive education, advocating for more innovative approaches to support diverse learning environments.

Enhanced Accessibility for Visually Impaired & Blind Artists (EAVIBA) using Machine Learning

This research paper introduces EAVIBA, an innovative voice-driven interface aimed at empowering individuals with visual impairments and blindness in the domain of art and creativity. Rooted in principles of accessible design, EAVIBA integrates advanced features including voice commands, text-to-speech functionality, and a customizable graphical user interface, exemplifying a commitment to inclusive technology. The paper delves into the intricacies of EAVIBA's development, elucidating its architectural framework, seamless integration of assistive technologies, and adherence to recognized accessibility standards such as the Web Content Accessibility Guidelines (WCAG). Employing a comprehensive approach, the research paper outlines the iterative process of user testing, providing both qualitative and quantitative insights into critical aspects such as interface clarity, voice command recognition accuracy, and the efficacy of customization options. The results of these evaluations affirm EAVIBA's remarkable success in delivering an inclusive, accessible, and empowering environment for individuals with visual impairments to engage in artistic expression, thereby contributing significantly to the broader landscape of assistive technologies and accessible interfaces. Keywords: Visually impaired, blind artists, voice-driven interface, assistive technologies, voice recognition, machine learning.

Dual-process theory-driven transparent approach for seniors to accept health misinformation detection results

Given seniors’ concerns about the reliability of black-box models in health misinformation detection (HMID), there is a pressing need for explainable HMID methods that provide transparency and instill trust. Explainable artificial intelligence (AI) aims to foster understanding and trust in AI models by implementing transparency that embodies explainability and interpretability. However, most existing explainable HMID methods solely provide post-hoc explanations and neglect the interpretation of the models’ internal logic, which prevents seniors from satisfactorily accepting the HMID results. Therefore, this study proposes a transparent Knowledge Graph-aware Two-Stage approach (KG2S) driven by the dual-process theory. KG2S combines explainability and interpretability by utilizing knowledge graphs (KGs) in two stages: Knowledge Breadth Retrieval (KBR) and Knowledge Depth Reasoning (KDR). These stages correspond with the heuristic and analytic processes of the dual-process theory, which encompasses human information processing. In the KBR stage, we leverage rich facts in KGs to replicate heuristic distillation behaviors of humans through a novel similarity-diversity twofold filter. In the KDR stage, we employ a hierarchical attention network to emulate humans’ coarse-to-fine knowledge analyses during decision-making. Extensive experiments were conducted on two real-world datasets, along with user testing, to assess the effectiveness of the proposed model. Results demonstrated that the proposed model not only outperformed competing methods in terms of HMID accuracy but also provided persuasive detection processes and reasons. Moreover, the model showed high adaptability to new topics in HMID. Our research offers valuable insights into integrating humanistic into AI algorithms and promoting the trustworthiness of AI systems.

AR based Indoor Navigation System

Indoor navigation within college campuses poses unique challenges due to complex building layouts and varying levels of accessibility. This paper presents a student project aimed at developing an Augmented Reality based indoor navigation system (AR-INS) specifically tailored for college campuses. The system integrates computer vision, depth perception and Google’s ARCore techniques to provide users with real-time navigation guidance using their smartphones or AR-enabled devices. We describe the system architecture, implementation details, and evaluation results from user testing conducted on our college campus. The project highlights the feasibility of utilizing AR technology for enhancing campus navigation experiences and provides valuable insights for future research and development in this domain. This project addresses the growing need for efficient indoor navigation, which can significantly improve the overall user experience and accessibility in indoor spaces. The AR-INS is not only user- friendly but also adaptable to various environments, making it a valuable addition to smart building infrastructure and enhancing the quality of life for users. The implementation of this system can have an impact on how people navigate complex indoor spaces, and give an solution for complex indoor spaces. Keywords—AR-INS, computer vision, depth perception Google’s ARCore

Evaluating a web-based guide for designing digital patient experiences: preliminary results of a user test with design students

AbstractWe created a web-based design guide to transfer our previous research findings to better support design education in the digital health design area for improving patient experience. To seek insights to iteratively improve the design guide, we conducted a workshop with 19 MSc students who specialized in design for healthcare. The guide was perceived as having the potential to improve their understanding of digital PEx improvements, but the content clarity and information presentation need to be improved.

Mastering the art of technical writing in it: Making complex things easy to understand in Atate campus

This research, conducted at the Atate Campus of Nueva Ecija University of Science and Technology, delves into the realm of technical writing within the Information Technology (IT) sector, focusing on the pedagogical strategies that facilitate the comprehension of complex technical topics by IT students. It seeks to uncover the challenges IT students face when engaging with technical documentation and the effective methods to surmount these obstacles. Through a qualitative and quantitative analysis involving a survey distributed to 100 IT students across various academic years, the study identifies significant challenges such as conceptual complexity, jargon prevalence, and the need for audience-tailored documentation. The research highlights best practices like the use of plain language, visuals, real-world examples, and the critical role of user testing in enhancing document clarity and understandability. Furthermore, it examines preferences in writing styles and document structures, offering insights into effective technical communication tailored to diverse audience needs. This study proposes a set of recommendations aimed at improving technical writing curriculum and pedagogy, encouraging future research, and fostering a deeper understanding of technical subjects among IT students through enhanced technical writing skills.

Optimized Transfer Learning Based Dementia Prediction System for Rehabilitation Therapy Planning

Abstract: Dementia is a progressive neurodegenerative disease leading to cognitive decline, and current medical interventions can only slow its progression. Early prediction of dementia onset holds potential for preventive measures. This study aims to develop a machine learning model using transfer learning from magnetic resonance imaging (MRI) data to predict dementia. The project employs k-fold cross-validation and various parameter optimization algorithms during model training to enhance prediction accuracy. The final voting classifier and optimized Transfer Learning algorithms achieved an impressive accuracy, surpassing the performance of competing methods on the same dataset. This suggests the efficacy of the proposed transferlearning approach in predicting dementia data. The developed model enables early diagnosis of dementia, a crucial factor in halting neurological deterioration associated with the disease. This is particularly relevant for regions with limited access to human physicians, providing a valuable tool for underserved populations. The proposed system holds promise for planning rehabilitation therapy programs for dementia patients. As a tool for early diagnosis and intervention, the model not only contributes to improved patient outcomes but also addresses the challenges faced by regions lacking access to sufficient healthcare resources. And also include ensemble methods (Voting Classifier, Stacking Classifier) and a hybrid CNN-LSTM approach are employed for improving accuracy, with the Voting Classifier achieving an impressive 100% accuracy. To facilitate user testing, a user-friendly front end using the Flask framework, incorporating user authentication, was proposed for seamless application of this advanced dementia prediction system.

The exercise-app Axia for axial spondyloarthritis enhances the home-based exercise frequency in axial spondyloarthritis patients - A cross-sectional survey.

Patients with axial spondyloarthritis (axSpA) benefit from regular home-based exercise (HbE). In spite of recommendations, a relevant proportion of German axSpA patients does not adhere to recommended HbE practices. To enhance HbE care, we developed the novel digital therapeutic (DTx) "Axia" compliant with the European medical device regulation (MDR). Axia offers a modern app-based HbE solution with patient educative content and further integrated features. We aimed to assess Axia's efficacy, attractiveness, and functionality through a survey among axSpA-patients involved in the first user tests. A mixed-method online questionnaire with 38 items was administered to 37 axSpA volunteers after using Axia. Numeric rating scales (NRS) and likelihood scales were primarily used. HbE frequency significantly increased from a median of 1day/week to 6 days/week (p < 0.001) by using Axia. Existing HbE practitioners also increased their frequency (median of 4 days/week before, 6 days/week with Axia, p < 0.05). Axia received a median rating of 5 out of 5 stars. On NRS scales, Axia scored a median of 9 for intuitiveness and design, and a median of 8 for entertainment. 64.9% reported improved range of motion, 43.2% reported reduced pain, and 93.6% enhanced disease-specific knowledge. All users recommended Axia to other patients. Axia increases axSpA patients HbE frequency, possibly due to its good intuitiveness and design, leading to reduction in pain and subjective improvement of range of motion. This warrants further investigation in large randomized controlled interventional trials to establish its efficacy conclusively and patients adherence to HbE.

Learning Abstraction Selection for Bayesian Program Analysis

We propose a learning-based approach to select abstractions for Bayesian program analysis. Bayesian program analysis converts a program analysis into a Bayesian model by attaching probabilities to analysis rules. It computes probabilities of analysis results and can update them by learning from user feedback, test runs, and other information. Its abstraction heavily affects how well it learns from such information. There exists a long line of works in selecting abstractions for conventional program analysis but they are not effective for Bayesian program analysis. This is because they do not optimize for generalization ability. We propose a data-driven framework to solve this problem by learning from labeled programs. Starting from an abstraction, it decides how to change the abstraction based on analysis derivations. To be general, it considers graph properties of analysis derivations; to be effective, it considers the derivations before and after changing the abstraction. We demonstrate the effectiveness of our approach using a datarace analysis and a thread-escape analysis.

Quantifying Dwell Time With Location-based Augmented Reality: Dynamic AOI Analysis on Mobile Eye Tracking Data With Vision Transformer.

Mobile eye tracking captures egocentric vision and is well-suited for naturalistic studies. However, its data is noisy, especially when acquired outdoor with multiple participants over several sessions. Area of interest analysis on moving targets is difficult because A) camera and objects move nonlinearly and may disappear/reappear from the scene; and B) off-the-shelf analysis tools are limited to linearly moving objects. As a result, researchers resort to time-consuming manual annotation, which limits the use of mobile eye tracking in naturalistic studies. We introduce a method based on a fine-tuned Vision Transformer (ViT) model for classifying frames with overlaying gaze markers. After fine-tuning a model on a manually labelled training set made of 1.98% (=7845 frames) of our entire data for three epochs, our model reached 99.34% accuracy as evaluated on hold-out data. We used the method to quantify participants' dwell time on a tablet during the outdoor user test of a mobile augmented reality application for biodiversity education. We discuss the benefits and limitations of our approach and its potential to be applied to other contexts.

A study of easy-to-read adaptations in climate communication: a corpus analysis

AbstractAccess to clear and scientifically verified information is crucial to understand the scale of the challenge of the climate crisis, allowing policymakers, governments, and individuals to better respond to it. However, for some, including people with cognitive disabilities as well as individuals who have difficulty reading, climate science can be difficult to understand with complex sentence structures and jargon obscuring important environmental information. The aim of this paper is to assess the main linguistic barriers to information about the climate crisis for people with cognitive disabilities and reading difficulties. Using the examples of the Greater London Authority’s Environment Strategy Executive Report and the Northern Ireland Executive Discussion Document on a Northern Ireland Climate Change Bill, and their adaptations into Easy-to-Read as our corpus, we will examine the different morphosyntactic and lexical elements of each text to determine their overall level of complexity, such as sentence structure or vocabulary. Our results show that both Easy-to-Read versions were generally easier to understand than the standard versions. However, the Northern Ireland Easy-to-Read text was more syntactically complex with longer and more complicated sentence structures, owing to its function as a consultation document. We conclude that the main access barriers in the Northern Ireland Easy-to-Read text are related to its morphosyntactic elements, a conclusion supported by our manual analysis of the standard text. More generally, the lexical aspects of both Easy-to-Read texts showed positive results, which suggests that they may be more easily simplified compared to morphosyntactic aspects. However, more systemic and contrastive research is required to confirm our findings, in addition to user testing with people with cognitive disabilities and individuals who have difficulty reading.