Usability Testing Research Articles

Evaluating the Usability of a Virtual Classroom Application from the Educator Perspective

The purpose of this research is to reveal the usability of virtual classroom application, which is used for synchronous lessons, in terms of effectiveness, efficiency, and satisfaction. A user-based approach was applied in order to collect quantitative data in this mixed method research. The effectiveness and efficiency of the system were revealed on the basis of 13 authentic tasks considering the status of the instructors performing the tasks, the number of critical mistakes they made during the tasks, and the time they spent on completing the tasks. The satisfaction of instructors with the virtual classroom application was determined through the SUS-satisfaction scale, observations during user tests, and interviews. A total of 12 instructors were in the data collection process. According to the findings, the completion rate of two of the seven tasks was over 90%. However, the successful completion rate of the total of six tasks was limited to 20%. It was also determined that the number of errors in the remaining 11 tasks exceeded the ideal number of operations and the completion periods were considerably longer. Overall results of the study revealed that the educator satisfaction levels for the Adobe Connect virtual classroom application were quite low.

Efficient Learning for hearing-impaired: Two Way Sign Language Translator using AI

This research paper describes the design and im- plementation of a two-way sign language translator for deaf- mute and hearing individuals. The device uses a hand gesture recognition system to translate sign language into text for hearing individuals and converts text into sign language videos for deaf- mute individuals. The paper goes over the technical challenges and solutions involved in creating a legitimate system that recognizes gestures and a sign language video generation system. The results of the user testing with deaf-mute and hearing individuals show that the device has high accuracy and is user-friendly. The two-way sign language translator has the possibility of enhancing communication and accessibility in daily interactions for deaf-mute people.

Enhancing data tables user experience via automated adaptations

User experience is a crucial aspect of software quality, and while various guidelines and heuristics have been proposed to enhance it, tables have not received as much attention as other elements. Additionally, many developers are unaware of the guidelines they should follow in this regard. The objective of this article is to enhance the user experience on websites by streamlining the development process of usable tables. To achieve this, we propose a set of heuristics integrated into a tool called BetterTable that enables automatic application of usability improvements while parsing the HTML document. The complexity involved in developing a table with applied usability guidelines is evaluated using different implementations: the proposed approach, DataTables, and AntDesign. Furthermore, tests are conducted to validate the effectiveness of the proposed heuristics through user interactions with 13 table usage scenarios, measuring time, mouse movement, and clicks. Each user is randomly presented with 13 scenarios that either meet or do not meet a specific guideline. The results indicate that our proposal reduces implementation complexity by 43.83% and 24.69% compared to DataTables and AntDesign, respectively. Moreover, user testing reveals that 6 out of the 13 guidelines show improvements in at least 2 of the 3 calculated metrics. Guidelines fulfilled show 40.37% less average completion time and 20.39% less mouse movement than the baseline. Based on the conducted studies, we observed that compliant tables, on average, exhibit reduced user time and mouse movements compared to non-compliant tables. However, no significant differences are found in terms of clicks.

ANALISIS USABILITY SISTEM MANAJEMEN PEMBELAJARAN DI WASKITA UNIVERSITAS AMIKOM: PENDEKATAN DENGAN SYSTEM USABILITY SCALE

This research was conducted at Amikom University Yogyakarta, a university that responded adaptively to the pandemic by developing the Waskita Learning Management System (LMS). Waskita LMS functions as a supporting platform for teaching and learning activities, facilitating the exchange of lecture materials, assignment collection, and quizzes between lecturers and students. Usability analysis was carried out using the System Usability Scale (SUS) method, a simple and reliable user testing method. The analysis results show that LMS Waskita received an average SUS score of 54.2, falling into the Not Acceptable category, indicating the need for improvement. Apart from calculating the SUS, an analysis of respondents' suggestions was also carried out to strengthen the evaluation results. This research provides valuable insights for developing and improving the quality of the Waskita LMS to improve user experience and usability.

Initiating and integrating user-centred design in upgrading the e-learning layout: a case study on FlexLearn (Brightspace) 2.0 template

PurposeThe purpose of this paper is to articulate how the user experience (UX) approach was initiated and integrated into the centre’s scope of operations with the objective of improving the e-learning layout on the D2L learning management system (LMS). One of the most effective ways to collect user feedback has historically been to evaluate user interfaces using strategies from user testing. The integration of a UX approach by the Centre for ODL Experiences (COLE) at Wawasan Open University has led to a more user-oriented design of FlexLearn by conducting user testing on students as the target users of the platform and gathering course leaders’ (CLs) feedback after the presentation of the new template.Design/methodology/approachSince the process of design and development is a looping process, the first user testing methods employed were observation and interviews, which were conducted over the course of numerous sessions. The data collection used a mixed-methods approach, combining quantitative demographic and background data with qualitative feedback from open-ended questions and real-time interview responses. A standardized questionnaire gathered demographic information, while questions for feedback forms and interviews were adjusted based on specific tasks to explore usability and user interactions comprehensively.FindingsThe findings revealed overall positive feedback, with some concerns highlighted by the students who claimed to have trouble navigating the courses during the initial prototype. In addition to the qualitative data from the user testing session, a quantitative method based on an online questionnaire was also utilised for the CLs after the presentation of the final layout. Positive responses were received from the CLs, and constructive suggestions were considered for FlexLearn 3.0.Research limitations/implicationsThis paper is among the first that articulates the process of initiating and integrating user-centred design in an effort to improve the user experience of online and ODL platforms and LMSs. It will contribute to a dialogue on investigating and prioritising learners’ ODL experiences to ensure education equity across all levels or categories of students, which aligns with the United Nations Sustainable Development Goals.Practical implicationsThe integration of UX and user testing allows us to better identify what users like, their concerns and their needs. We gain important input on how easy or difficult it is to use the system, move around it and how much they enjoy using it. This feedback helps us make changes to the design so that the final product is more in line with what users want. It also allows us to discover problems before they become major, saving time and effort later on. Finally, integrating user input improves the LMS, delivering a more fun and successful learning experience for everyone.Social implicationsUser-friendly systems arise as institutions prioritise user-centred design, breaking down barriers for various learners. This develops an innovative culture, improving present learning experiences and setting a precedent for future generations. The emphasis on user demands helps to create a more accessible, adaptive and egalitarian educational landscape by connecting education with current technological trends. As education becomes more inclusive, the broader community benefits, emphasising the beneficial social impact of LMS user testing.Originality/valueBy articulating the process of integrating user testing on an LMS/e-learning prototype, helps us understand what users like, where they face problems and what needs improvement. By involving users in testing, we get valuable feedback on how easy it is to use the system, navigate around and overall, how much they enjoy using it. Case studies like this also offer universities concrete examples of real-world challenges and successes.

Development and Utility of an Interactive Online Dashboard for Monitoring Data Collection and Data Processing During a School-Based Health Survey.

We built an interactive online dashboard using Google Looker Studio to monitor data collection and data processing activities during the Adolescent Health Survey (AHS) 2022, a large-scale nationwide survey conducted among school-going adolescents in Malaysia. Through user testing and training, refinements were made to the initial dashboard, resulting in a more streamlined and concise dashboard design. The dashboard comprised 2 pages that provided key metrics on the progress of data collection and data processing, respectively. The introduction of the dashboard enhanced the quality and ease of weekly progress reporting during meetings of the survey's central coordinating team, while its drill-down and filtering functionalities helped us detect arising issues early and supported collaborative problem-solving. Research teams coordinating comparable school-based health surveys are invited to duplicate the dashboard using Looker Studio's built-in "Make a copy" function and customize it further based on their country- or survey-specific requirements.

A Case Study on (EAVIBA) Enhanced Accessibility for Visually Impaired & Blind Artists using Machine Learning

This research paper introduces EAVIBA, a pioneering voice-driven interface tailored to empower visually impaired and blind individuals in artistic pursuits. Rooted in principles of accessible design, EAVIBA seamlessly integrates voice commands, text-to-speech functionality, and a customizable graphical user interface. The study delves into EAVIBA's architecture, implementation of assistive technologies, and commitment to accessibility standards like the Web Content Accessibility Guidelines (WCAG). Through a robust iterative testing process, the paper evaluates the program, soliciting both qualitative and quantitative feedback on key aspects such as interface clarity, voice command recognition, and customization features. Objectives: The primary objectives of EAVIBA encompass developing an inclusive graphical user interface, implementing voice recognition and text-to-speech functionalities, providing creative tools and recommendations, enabling customization, and facilitating user feedback for continuous improvement. Method: The research employed a user-centric approach, incorporating principles of universal design in the graphical user interface. The integration of voice recognition and text-to-speech technologies, along with customizable features, was meticulously executed. Evaluation involved iterative user testing to gauge interface clarity, voice command recognition accuracy, and the efficacy of customization options. Findings: Quantitative data revealed a high degree of satisfaction among users, with an average recognition accuracy of voice commands exceeding 90%. Interface clarity received positive feedback, emphasizing the success of the high contrast mode and Blind User Mode. Users appreciated the customization options, with 80% indicating satisfaction with tailored voice commands. Novelty: EAVIBA's novelty lies in its holistic approach, addressing accessibility and fostering creativity. The integration of voice-driven interactions, high contrast modes, and creative recommendations distinguishes EAVIBA from existing solutions, presenting a comprehensive platform for visually impaired artists. Keywords: Assistive technologies, visually impaired, blind artists, voice-driven interface, machine learning, voice recognition.

Fluttering into Fluent English: Building an Interactive Voice-Based AI Learning App for Language Acquisition

The rapid progress in technology has transformed the way people learn languages, offering students innovative tools to improve their skills. SpeakBuddy is built using the Flutter framework and utilizes Azure Speech AI for speech functionalities. Speak Buddy incorporates various features, such as interactive conversation practice, detailed lessons covering vocabulary, grammar, and pronunciation, fun games, and AI-powered voice assistance. The app's creation involved several design cycles where feedback from language experts and user testing was used to ensure it was easy to use and effective. Continuous assessments were conducted to evaluate the app's usability, performance, and impact on language learning outcomes. The results showed high levels of satisfaction among users, who reported feeling more confident and proficient in spoken English after using the app. A key feature of Speak Buddy is its ability to provide instant feedback on pronunciation, helping students improve their speaking skills.

Hand Gesture Vocaliser using IoT

Abstract: This study suggests creating a hand gesture vocalizer system for people who are paralysed or deaf and cannot talk. By using the Internet of Things (IoT) technology, this system is able to recognise hand motions and convert them into voice, facilitating effective communication between users. A glove with sensors to identify hand movements and a microprocessor to process the data make up the system. After that, the data is sent wirelessly to a computer or mobile device, where a machine learning algorithm interprets the user's motions and turns them into audible speech. For those who are unable to communicate verbally, the suggested approach is anticipated to offer an efficient and natural means of doing so, enhancing their socialisation and quality of life. User testing will be used to assess the correctness and dependability of the system, and subsequent development efforts will concentrate on enhancing the system's portability and usability.

Enhancing Interactive IVR Using NLP

Abstract: This paper suggests a fresh approach to IVR design by incorporating natural language processing (NLP) techniques, enhancing user experiences. Traditional IVRs often frustrate users due to their rigid menu structures. By integrating NLP, we aim to create dynamic and interactive IVR systems, leading to increased user satisfaction, streamlined interactions, and improved business performance. Our self-deployment and user testing demonstrate the system's effectiveness in providing personalized and responsive support, transforming traditional IVRs into engaging communication channels.

Classical music recommendation algorithm on art market audience expansion under deep learning

Abstract The purpose of the study is to help users know about their favorite music and expand art market audiences. First, the personalized recommendation data of classical music are obtained based on the deep learning recommendation algorithm technology, artificial intelligence, and music playback software of users. Second, a systematic experiment is conducted on the improved recommendation algorithm, and a classical music dataset is established and used for model training and user testing. Then, the network model of the classical music recommendation algorithm is constructed through the typical convolutional neural network model, and the optimal parameters suitable for the model are found. The experimental results show that the optimal value of the dimension in the hidden layer is 192, and 24,000 training rounds can converge to the global optimum when the learning rate is 0.001. The personalized recommendation is provided for target users by calculating the similarity between user preference and potential features of classical music, relieving the auditory fatigue of art market audiences, improving user experience, and expanding the art market audience through the classical music recommendation system.

Decoding Gender Representation and Bias in Voice User Interfaces (VUIs)

This article offers a thorough exploration of gender representation and bias within Voice User Interfaces (VUIs), delving into their intricate impact on technology design, user interaction, and broader societal dynamics. It scrutinizes the entrenched gender stereotypes inherent in VUI design, revealing the intricate interplay between technology, cultural norms, and gender expectations[1]. It highlights the ethical implications of such biases, emphasizing the need for diverse perspectives in the development of VUIs[2], [3]. the article advocates for a multifaceted approach to understanding and identifying bias in VUIs, incorporating methodologies such as data analysis, algorithmic auditing, and user testing[4]. Beyond individual interactions, it discusses how gender bias in VUIs can erode trust, diminish user satisfaction, and perpetuate systemic inequalities[1]. Proposing a shift towards gender-neutral design principles, it advocates for inclusivity and equity, championing personalized experiences and diverse representation. Looking ahead, the article outlines future directions in VUI technology aimed at advancing gender equality through enhanced natural language understanding, sentiment analysis, and collaborative design approaches[4]. By unraveling and addressing gender representation and bias in VUIs, this article lays a robust foundation for fostering a more inclusive and equitable landscape in voice technology.

Probability maps for deep learning-based head and neck tumor segmentation: Graphical User Interface design and test

BackgroundThe different tumor appearance of head and neck cancer across imaging modalities, scanners, and acquisition parameters accounts for the highly subjective nature of the manual tumor segmentation task. The variability of the manual contours is one of the causes of the lack of generalizability and the suboptimal performance of deep learning (DL) based tumor auto-segmentation models. Therefore, a DL-based method was developed that outputs predicted tumor probabilities for each PET-CT voxel in the form of a probability map instead of one fixed contour. The aim of this study was to show that DL-generated probability maps for tumor segmentation are clinically relevant, intuitive, and a more suitable solution to assist radiation oncologists in gross tumor volume segmentation on PET-CT images of head and neck cancer patients. MethodA graphical user interface (GUI) was designed, and a prototype was developed to allow the user to interact with tumor probability maps. Furthermore, a user study was conducted where nine experts in tumor delineation interacted with the interface prototype and its functionality. The participants’ experience was assessed qualitatively and quantitatively. ResultsThe interviews with radiation oncologists revealed their preference for using a rainbow colormap to visualize tumor probability maps during contouring, which they found intuitive. They also appreciated the slider feature, which facilitated interaction by allowing the selection of threshold values to create single contours for editing and use as a starting point. Feedback on the prototype highlighted its excellent usability and positive integration into clinical workflows. ConclusionsThis study shows that DL-generated tumor probability maps are explainable, transparent, intuitive and a better alternative to the single output of tumor segmentation models.

Learning Video Innovation Based on Pancasila Student Profiles: Exploration of the Values of Historical Places in Margarana

Learning media is a tool or infrastructure used in learning activities and is useful in helping the teaching and learning process and increasing students' motivation to learn. This research aims to develop video media for teaching values based on Pancasila student profiles at the Margarana historical site for elementary school students. The test subjects for this research were material/content experts, design experts, learning media experts, and 28 students. This research uses the ADDIE development model, questionnaire and test data collection methods, quantitative descriptive data analysis techniques, and t-test inferential statistical analysis. The research results found that the design was a learning video and obtained a score of 90.90% with very good qualifications. The results of the expert test and user test showed that the media content expert test results obtained a score of 93.18%, the results of the learning design expert test obtained a score of 90%, the learning media expert test results obtained a score of 91.66%, the individual test results obtained a score of 90.96%, and the small group test results obtained a score of 90.00%, so this learning video received very good qualifications. It is worthy of use. The product effectiveness results were obtained from the t-test statistical test, namely t-count = 6.953. For db = 27 and a significance level of 5%, the t-table figure = 1.703. So, t-count > t-table, H0 is rejected, and H1 is accepted, meaning there is a significant difference in science learning outcomes before and after video learning media. Thus, the learning video media in this research is effectively used in the science content of my area material.

Physicians found an interactive tool displaying structured evidence summaries for multiple comparisons understandable and useful: a qualitative user testing study

ObjectivesTo evaluate and improve “Making Alternative Treatment Choices Intuitive and Trustworthy” (MATCH-IT)—a digital, interactive decision support tool displaying structured evidence summaries for multiple comparisons—to help physicians interpret and apply evidence from network meta-analysis (NMA) for their clinical decision-making. Study Design and SettingWe conducted a qualitative user testing study, applying principles from user-centered design in an iterative development process. We recruited a convenience sample of practicing physicians in Norway, Belgium, and Canada, and asked them to interpret structured evidence summaries for multiple comparisons—linked to clinical guideline recommendations—displayed in MATCH-IT. User testing included (a) introduction of a clinical scenario, (b) a think-aloud session with participant–tool interaction, and (c) a semistructured interview. We video recorded, transcribed, and analyzed user tests using directed content analysis. The results informed new updates in MATCH-IT. ResultsDistributed across 5 development cycles we tested MATCH-IT with 26 physicians. Of these, 24 (94%) reported either no or sparse prior experience with interpretation of NMA. Physicians perceived MATCH-IT as easy to interpret and navigate, and appreciated its ability to provide an overview of the evidence. Visualization of effects in pictograms and inclusion of information on burden of treatment (“practical issues”) were highlighted as potentially useful features in interacting with patients. We also identified problems, including undiscovered functionalities (drag and drop), suboptimal tutorial, and cumbersome navigation of the tool. In addition, physicians wanted definition/explanation of key terms (eg, outcomes and “certainty”), and there were concerns that overwhelming evidence from a large NMA would complicate applicability to clinical practice. This led to several updates with development of a new start page, tutorial, updated user interface for more efficient maneuvering, solutions to display definition of key terms and a “frequently asked questions” section. To facilitate interpretation of large networks, we improved categorization of results using color coding and added filtering functionality. These modifications allowed physicians to focus on interventions of interest and reduce information overload. ConclusionThis study provides proof of concept that physicians can use MATCH-IT to understand NMA evidence. Key features of MATCH-IT in a clinical context include providing an overview of the evidence, visualization of effects, and the display of information on burden of treatments. However, unfamiliarity with the Grading of Recommendations Assessment, Development and Evaluation concepts, time constraints, and accessibility at the point of care may be challenges for use. To what extent our results are transferable to real-world clinical contexts remains to be explored.

Developing a Hair Styling Flipbook in the Television Industry for West Jakarta R Hairdressers

The research aimed to 1) develop a hairdressing flipbook in the television industry for hairdressers in the West Jakarta area, and 2) develop a hairdressing flipbook that is appropriate for use by hairdressers in the television industry in the West Jakarta area. The method used was Research and Development (R&D) with the 4D development model: define, design, develop, and disseminate. The product dissemination employed qualitative and quantitative descriptive analysis. The research findings are elaborated as follows. (1) The design used in developing the flipbook shows talent hair styling in the television industry such as hair styling materials and equipment, hair styling for male and female talents, and hair care, the hair styling flipbook which can be accessed via the file link: https://online.flipbuilder.com/xnfly/axem/ (2) know the level of feasibility of developing this flipbook, the user test calculation validation assessment is 91%. The conclusion is that the "development of a Hairdressing Flipbook in the Television Industry for Hairdressers in West Jakarta " is highly feasible.

AI-Enhanced Accessibility Solutions using Chatbot

The "AI-Enhanced Accessibility Solutions" project addresses the real-world challenge of enhancing accessibility for individuals through advanced technologies. Utilizing React.js, Next.js, Tailwind, and AI APIs, this study focuses on generating inclusive content, including text, images, videos, code, and chatbot interactions. The primary objectives are to improve user interfaces, employ AI-driven content generation techniques, and ensure compliance with accessibility standards. The platform allows users to input self-generated prompts to produce customized data, such as text, images, videos, and code, tailored to their needs. An interactive AI-powered chatbot assists users in daily activities, adapting responses based on the quality of prompts provided, ensuring personalized and effective results. Key features include AI-generated content designed for various abilities. This includes alternative text for images, simplified text for cognitive accessibility, and adaptive chatbot responses. The project also focuses on generating accessible code, adhering to best web development practices. Tailwind CSS is used to optimize content presentation, addressing factors like color contrast and readability. The study includes a comparative analysis of accessibility features in popular apps versus those generated by our system. User testing and feedback mechanisms are employed to validate the project's effectiveness in meeting diverse user needs. This project not only advances AI-driven content creation but also represents a significant step towards creating inclusive digital environments. It emphasizes the importance of accessibility in technology, ensuring that digital content is accessible to all individuals, regardless of their abilities. Keywords: AI-Enhanced Accessibility, Inclusive Content, React.js, Next.js, AI APIs, User Interfaces, Accessibility Standards, AI-Powered Chatbot, Cognitive Accessibility

ARCHIGENIUS - VR MAZE SOLVING USING A* ALGORIHTM

Game is one of the entertainment media that is rapidly growing. On this day, playing games is one way to spare time for refreshing. One of the popular games this year is an action game such as First-Person Shooter (FPS). Besides, the FPS game genre has developed rapidly to make it more interesting for the player. Non-player Character (NPC) has an essential function in artificially and automatically to make a game more immersive and consisting of various kinds. Also, from observation, adding challenges when playing, namely enemy characters, can make a player more enjoyable. Besides, one of modern technology to develop innovations for supporting a game is called virtual reality. So, in this research, the FPS game is built by adding NPC elements with virtual reality technology. Also, the gameplay in this game is a survival maze. Based on the evaluation, this game application can display objects in a virtual reality smoothly, and all feature in this game can display and run effectively. Besides, the respondents also give good feedback for user testing which has average score 4.66 from several users. So, the conclusion from this research is this game can make the FPS game more interesting for users by adding NPC, 3D element, and virtual reality. Keywords - First-person Game, Virtual Reality, Survival Maze, Non-Player Character, virtual maze environment.

Unity-based Chinese Digital Education Game: Journey to Cultural Relics of Ancient China

This project is the study of a digital educational game primarily designed to aid foreigners in the process of learning Chinese characters and Chinese culture. The game was created to be three-dimensional, incorporating semi-realistic traditional Chinese art-style illustrations. In addition to the incorporated card memory game, designed to help memorize Chinese characters, there are also elements used to introduce the culture and history of China's four great inventions, assimilating them into four task-setting mini-games. Through this method, players can deepen their impression of Chinese culture by playing mini-games. The game was developed based on the Unity development platform, utilizing Maya for modeling, Substance Painter for textures, employing PS and Proceate for material painting, and interface design. From the interview results of the user test, the game has achieved the purpose of helping players learn Chinese characters, understand Chinese culture, and increase their interest in China.

CHATBOT USING LARGE LANGUAGE MODEL

The concept of Natural Language Processing has seen a remarkable advancement in the recent years. This remarkable advancement was particularly with the development of Large Language Models (LLM). Large Language Models are used to develop a human like conversations. This LLM is a part of Natural Language Processing which focuses on enabling computers to understand, interpret, and generate human language. The existing system of chatbots does not generate human like responses. The proposed system of chatbots uses the power of Large Language Models to generate more human like responses, providing the conversation in a natural way. By genereating human like respones, it will be in a natural way for the user. To enhance user experience, the chatbot uses a dynamic learning mechanism, by which it continuously adapt to user preferences and evolving conversational patterns. This system uses feedbacks from the users to refine its responses everytime.Moreover, the chatbot is designed with a multi-turn conversational context awareness, allowing it to maintain coherence and relevance throughout extended dialogues.The effectiveness of the proposed chatbot is evaluated through user testing, comparing its performance against traditional rule-based chatbots and existing conversational agents. This report explains about the usage of Large Language Models in the design and implementation of conversational chatbots. The outcomes of this research contribute to the advancement of intelligent chatbot systems, demonstrating the potential of large language models to significantly enhance conversational AI applications.