Usability Testing Methods Research Articles

Evaluation of Website Performance and Usability Using GTMetrix, Usability Testing, and System Usability Scale (SUS) Methods

This study was conducted to measure the performance of IAIN Metro's website in terms of performance and user perception to ensure that the campus website can adequately support visitors' needs. This study aims to determine things that need to be improved to improve the performance of the IAIN Metro website. To get comprehensive results from the performance of the website, this research uses GTMetrix to analyse the technical performance of the IAIN Metro website, and then, to test user perceptions, researchers use Usability Testing and System Usability Scale methods. In usability testing, several aspects will be measured to determine usability problems, namely learnability and efficiency, while the System Usability Scale questionnaire will be used to test the satisfaction level. Based on the test results using GTMetrix, after testing, several aspects of the access speed of the IAIN Metro website need to be improved. Although, in general, from the test results, Usability Testing and System Usability Scale users still consider the performance of the website to be acceptable, the results of the first task on Time Based Efficiency testing show that initial access to the main page metrouniv.ac.id, takes a relatively long time compared to other tasks. This is also evident from the GTMetrix score on the performance aspect, which has a low presentation of 25%. Therefore, optimisation is needed on the main page to improve website performance.

PERANCANGAN UI/UX APLIKASI MOBILE TIKET KONSER “EuFo” MENGGUNAKAN METODE DESIGN THINKING

EuFo is an app that focuses on the purchase of concert tickets, concert news, promotions, and other information. The distribution process of entertainment content, especially concerts, has been transformed by technological advances in the digital entertainment industry. After conducting User Research, the main problems were age filtering for underage users and user difficulties when entering the venue and also complicated and slow validation of concert tickets. Design applications using the Design Thinking method and analyze application performance results using the Usability Testing method as the goal of research. Design Thinking is an innovative approach to problem-solving that focuses on a thorough understanding of the user, creating creative solutions, and repeating iterations to the best results. The study combines components such as Empathize, Define, Ideate, Prototype, and Test. The results showed that the prototype of the concert ticket application was designed with a one-time iteration of changes and tested to ensure that the UI/UX design results matched the primary purpose. In the first and second tests, there's progress between the one and the two. Results in two tests with fifteen informants resulted in a comparison of the results of two tests: direct success 7%, mission unfinished 0%, misclick rate 5% and duration 10s.

Usability testing of a conceptual model through retrospective cross-case analysis

Qualitative conceptual models are commonly used in the scientific literature to make complex phenomena easier to understand. However, the effectiveness and usability of conceptual models to serve as analytical tools is rarely explored and tested, and there is a lack of guidelines for such analyses. This paper adapts and combines the methods of usability testing and cross-case analysis to describe a systematic protocol to facilitate the qualitative evaluation of conceptual models. Usability testing is an established method for identifying problems or shortcomings within a product and for assessing different dimensions of product usability: suitability, accessibility, relevance, and integrity. Cross-case analysis, on the other hand, is a qualitative research method for systematically comparing information from individual case studies and identifying commonalities and patterns that apply across cases. Taken together, these methodological approaches provide a structured way of retrospectively applying a conceptual model to existing literature and thereby evaluating its effectiveness in meeting its intended purpose.•We show how researchers can prepare, conduct, and synthesise the results of a usability test of a conceptual model.•We provide recommendations for the practical implementation of each step.•We outline the benefits, limitations, and ethical considerations that researchers should be aware of.

Realizing the Full Potential of Clinical Decision Support: Translating Usability Testing into Routine Practice in healthcare operations.

Clinical Decision Support (CDS) tools have a mixed record of effectiveness, often due to inadequate alignment with clinical workflows and poor usability. While there's a consensus that usability testing methods address these issues, in practice, usability testing is generally only used for selected projects (such as funded research studies). There is a critical need for CDS operations to apply usability testing to all CDS implementations. In this State of the Art / Best Practice paper, we share challenges with scaling usability in healthcare operations and alternative methods and CDS governance structures to enable usability testing as a routine practice. We coalesce our experience and results of applying guerilla in-situ usability testing to over 20 projects in 1 year period with the proposed solution. We demonstrate the feasibility of adopting "guerilla in-situ usability testing" in operations and their effectiveness in incorporating user feedback and improving design. Although some methodological rigor was relaxed to accommodate operational speed, the benefits outweighed the limitations. Broader adoption of usability testing may transform CDS implementation and improve health outcomes.

Development, validation, user testing of patient information leaflet to improve awareness of metabolic side effects in patients on atypical antipsychotics

BackgroundMetabolic adverse effects of second-generation antipsychotics (SGAs) have been well-documented in literature. This study aimed to develop, validate, and evaluate the usefulness of patient information leaflet (PIL) by user testing method to improve metabolic monitoring for psychiatric patients prescribed with SGAs. MethodsThe study included out-patients above 18 years of age diagnosed with schizophrenia, schizoaffective, bipolar disorders prescribed with SGAs. Database such as micromedex, guidelines on developing PIL and factsheets on antipsychotics were used to prepare the content of the leaflet and was given to experts for validation. The PIL covered information on definition of antipsychotic and its types, metabolic syndrome and its prevention such as lifestyle modifications and regular monitoring. Readability of the PIL was assessed using Flesch reading ease (FRE), Flesch Kincaid grade level (FK-GL) and user testing in 20 similar patient population. ResultsThe BALD scores on layout and design of the PILs was found to be 29. The FRE readability test score was found to be 63.5 and FKGL test score was found to be 7.6. Among the 20 patients, 5 of them favoured English PILs, while 15 chose Kannada PILs. The overall pre and post user-testing mean scores for knowledge assessment significantly increased from 60.62 ± 23.39 to 91.25 ± 11.54. Overall, about 85 % of patients gave the PIL leaflets a good rating. ConclusionPatient education materials of high quality that are created with pictures may aid in a better knowledge of the treatment for psychiatric patients who are on long term SGA therapy.

Penerapan Model Design Thinking Pada Perancangan Aplikasi Informasi Desa Wisata Kabupaten Bantul

Tourism actors in the current technological era have implemented information systems. With the rapid growth of tourist villages in Bantul Regency, there is a need for promotion and digital information delivery media. However, in developing digital media it is also necessary to pay attention to aspects of the users who are the target market. The design of the application called sidewi mobile (mobile tourist village information system) is based on user experience and needs, using the Design Thinking methodology which has five stages as follows: Empathize, Define, Ideate, Prototype, and Test. The design of the Sidewi mobile application was created using FIGMA software. This research has direct benefits, namely that it can be used as a benchmark for design needs before the development process. The results of the design are then tested using the usability testing method. Using a user friendly design approach and conducting testing using usability testing with the results of five users being able to complete the testing proves that when it was created using user experience there were no significant difficulties when used and it covered all needs.

Pengembangan UI/UX Sistem Pengelolaan Inventori Menggunakan Metode Design Thinking

System development requires an interface design that suits the user, this is so that the system can be used fully without reducing or ignoring its main functions. Dwi Jaya Store which operates in the wholesale and retail product sales sector faces challenges such as managing suppliers which may be more than one supplier, managing product orders to suppliers based on inventory levels, managing product returns to suppliers if there are defects or damage, managing the traffic of goods from main warehouse to shops and report management. Dwi Jaya Store requires an inventory system that is easy to use but does not reduce the performance and function of the system. This research aims to produce a prototype inventory system that users use easily. This research uses the design thinking method, usability testing method, and Figma as a design tool. This research produced a prototype with a usability testing value of 78.4% which was included in the Good Application category.

UX Analysis on SpeedID Application Using Usability Testing Method and System Usability Scale

The SpeedID application is a smart city application developed by a subsidiary of PT Bamboomedia Cipta Persada, namely PT Inovasi Solusi Nusantara since 5 years ago. The SpeedID application wants to present a solution to the city's problems to become a new digital identity for the smart city community. Because it was only developed 5 years ago, the SpeedID application is classified as a new application and has never been analyzed for usability before. Usability analysis is carried out to improve user experience, so that the SpeedID application can be accepted and used more easily by users. This research uses Usability Testing method with Performance Measurement and Retrospective Think Aloud (RTA) techniques and System Usability Scale (SUS). The results obtained are the SpeedID application has a quality that cannot be said to be effective, efficient and meet user satisfaction. In addition, the average score of the System Usability Scale (SUS) Questionnaire is 70.33. The score is rated "C" with an adjective rating of "Good" with the acceptance range included in the "Marginal" category, and finally the net promotion score (NPS) is included in the "Passive" category, which explains that the use of the SpeedID application gets an assessment that is still marginalized by its users. This shows that the SpeedID application still urgently needs correction to improve quality to its users and design improvements also need to be made so that the SpeedID application is even better at meeting user expectations in the future.

ANALISIS USABILITY SISTEM MANAJEMEN PEMBELAJARAN DI WASKITA UNIVERSITAS AMIKOM: PENDEKATAN DENGAN SYSTEM USABILITY SCALE

This research was conducted at Amikom University Yogyakarta, a university that responded adaptively to the pandemic by developing the Waskita Learning Management System (LMS). Waskita LMS functions as a supporting platform for teaching and learning activities, facilitating the exchange of lecture materials, assignment collection, and quizzes between lecturers and students. Usability analysis was carried out using the System Usability Scale (SUS) method, a simple and reliable user testing method. The analysis results show that LMS Waskita received an average SUS score of 54.2, falling into the Not Acceptable category, indicating the need for improvement. Apart from calculating the SUS, an analysis of respondents' suggestions was also carried out to strengthen the evaluation results. This research provides valuable insights for developing and improving the quality of the Waskita LMS to improve user experience and usability.

Initiating and integrating user-centred design in upgrading the e-learning layout: a case study on FlexLearn (Brightspace) 2.0 template

PurposeThe purpose of this paper is to articulate how the user experience (UX) approach was initiated and integrated into the centre’s scope of operations with the objective of improving the e-learning layout on the D2L learning management system (LMS). One of the most effective ways to collect user feedback has historically been to evaluate user interfaces using strategies from user testing. The integration of a UX approach by the Centre for ODL Experiences (COLE) at Wawasan Open University has led to a more user-oriented design of FlexLearn by conducting user testing on students as the target users of the platform and gathering course leaders’ (CLs) feedback after the presentation of the new template.Design/methodology/approachSince the process of design and development is a looping process, the first user testing methods employed were observation and interviews, which were conducted over the course of numerous sessions. The data collection used a mixed-methods approach, combining quantitative demographic and background data with qualitative feedback from open-ended questions and real-time interview responses. A standardized questionnaire gathered demographic information, while questions for feedback forms and interviews were adjusted based on specific tasks to explore usability and user interactions comprehensively.FindingsThe findings revealed overall positive feedback, with some concerns highlighted by the students who claimed to have trouble navigating the courses during the initial prototype. In addition to the qualitative data from the user testing session, a quantitative method based on an online questionnaire was also utilised for the CLs after the presentation of the final layout. Positive responses were received from the CLs, and constructive suggestions were considered for FlexLearn 3.0.Research limitations/implicationsThis paper is among the first that articulates the process of initiating and integrating user-centred design in an effort to improve the user experience of online and ODL platforms and LMSs. It will contribute to a dialogue on investigating and prioritising learners’ ODL experiences to ensure education equity across all levels or categories of students, which aligns with the United Nations Sustainable Development Goals.Practical implicationsThe integration of UX and user testing allows us to better identify what users like, their concerns and their needs. We gain important input on how easy or difficult it is to use the system, move around it and how much they enjoy using it. This feedback helps us make changes to the design so that the final product is more in line with what users want. It also allows us to discover problems before they become major, saving time and effort later on. Finally, integrating user input improves the LMS, delivering a more fun and successful learning experience for everyone.Social implicationsUser-friendly systems arise as institutions prioritise user-centred design, breaking down barriers for various learners. This develops an innovative culture, improving present learning experiences and setting a precedent for future generations. The emphasis on user demands helps to create a more accessible, adaptive and egalitarian educational landscape by connecting education with current technological trends. As education becomes more inclusive, the broader community benefits, emphasising the beneficial social impact of LMS user testing.Originality/valueBy articulating the process of integrating user testing on an LMS/e-learning prototype, helps us understand what users like, where they face problems and what needs improvement. By involving users in testing, we get valuable feedback on how easy it is to use the system, navigate around and overall, how much they enjoy using it. Case studies like this also offer universities concrete examples of real-world challenges and successes.

GEOGRAPHIC INFORMATION SYSTEM MAPPING BANTEN MAKING SERVICES IN DENPASAR CITY

The majority of Bali's population is Hindu. Hinduism is a religion that uses offerings as a means to carry out prayers. Currently, women in Bali in particular who usually have the task of making offerings are already actively working outside the home all day so they don't have much time to make offerings as a means of ceremony so it doesn't rule out the possibility of buying offerings according to their needs, there is a lack of information regarding location and information regarding The existence of offerings making services, especially in Denpasar City as the capital of Bali province, is a problem being faced. This lack of information can cause difficulties for people who actively work outside the home to provide the financial aid needed for ceremonies. Initial research was carried out by distributing several questions to residents of the city of Denpasar which were given randomly. The results of this research stated that around 90% of the community really needed information about banten making services due to the lack of time in the process and the complexity of making banten which is used as a means of appropriate ceremonies. each other's needs. Based on these problems, it is necessary to develop an internet-based Geographic Information System for Mapping Banten Manufacturing Services in Denpasar City using internet-based Geographic Information System software, namely Google My Maps so that it can be accessed online by the people of Denpasar in particular. After this system is built, a trial will be carried out on the community to assess the feasibility of the software in the form of a questionnaire using the Guttman scale, data analysis using the feasibility percentage technique. The research method uses field methods (field research). Product trials use the usability testing method, user trials use the feasibility percentage formula. It is hoped that the resulting digital map will help the people of Denpasar city to easily obtain information about the services for making offerings that are spread across so that problems in providing offerings for ceremonies can be made available quickly without having to take time off work.

System interactive reader using eye-tracker technology in ebook reader

Interest in using ebooks by the academic community is very high. Still, there is a problem when readers are reading through screens, tend to read fast, only scan the necessary parts, and don't focus on paying attention to the content they read, so this reduces the quality of reading because readers don't study the overall meaning of the sentence. Hence, this research aims to build an interactive reader system by integrating eye tracker technology with a webcam which is expected to solve the problem of decreasing the quality of reading through the screen by helping readers stay focused on their reading and providing an interactive system that makes it easier for readers to control the computer while reading. This research adopts the waterfall method and is divided into six stages. The system is designed using class diagrams, use case diagrams, and activity diagrams. Also, the system is built using the Python language with the Django framework. Then, the interactive reader system was tested using black box testing and usability testing methods. Based on the test results, it is shown that the interactive reader system that was built can help improve the quality and concentration when reading activities take place.

IDENTIFYING AND FIXING UX-FRICTION EMPLOYEE DEVELOPMENT WEBSITE WITH MIXED APPROACH HEART FRAMEWORK AND USABILITY TESTING

The company website has an important role in the business processes that occur in the company, one of which is managing employee productivity. PT.XYZ utilizes the Kaizen website to improve the quality of its employees. Assessment of user experience while operating the website can be a form of evaluation between expectations and actual conditions related to website feasibility. In this study, the researcher intends to find out what indicators are obstacles in the operation of a web-based application called the Kaizen website at PT.XYZ so that there is inefficiency in its use. This research involved 54 respondents who were employees of PT.XYZ. The combination of HEART and Usability Testing methods is used to identify obstacles experienced by users and evaluate user experience when using the Kaizen website. The results of the analysis are then used to improve website performance in order to create a good user experience. The results showed that the HEART score increased by an average of 9% on each variable. Usability testing assessment also shows that the Kaizen website reaches the acceptable category with a score of 90.5 after improvements are made.

Perancangan Ulang Desain UI/UX Website Digilib Universitas Sriwijaya Menggunakan Metode Double Diamond

Perancangan Ulang Desain UI/UX Website Digilib Universitas Sriwijaya Menggunakan Metode Double Diamond

Analysis of student responses toward activities using e-learning in higher education

The aim of this study is to investigate how students respond to learning activities using e-learning. Researchers analyzed students' responses based on four aspects of activity, including attendance discipline, assignment collection accuracy, enthusiastic use of e-learning teaching materials, and participation in group discussions. The research method used in this study is a quantitative descriptive method and usability testing method. The data analysis technique used is descriptive percentage. The sample for this study consisted of 100 students from the informatics education study program, including the batches of 2019, 2020, and 2021. The results showed that the average response rate of students towards e-learning activities was 67.19% for the year 2019, 81.32% for the year 2020, and 78.59% for the year 2021. Based on the findings, it can be concluded that students from all three batches responded positively to the e-learning activities.

Team usability testing: development and validation of a groupware usability evaluation method

As virtual teamwork became widespread, the importance of the usability of groupware has increased, as groupware must have high usability in order to properly support teamwork. Several studies have focused on the usability evaluation of groupware, yet, both in terms of science and practice, there is a lack of a method that, like existing empirical single-user methods, can be routinely applied in software development practice. The new Team Usability Testing method differs from previous methods in that it is an empirical method for evaluating synchronous (real-time) collaborative software that explores the team usability problems with the help of real or potential users. The method consists of questionnaires, screen recording videos and group interviews, and the framework of data processing is based on the theory of the mechanics of collaboration. In the four steps of the development of the method, empirical studies under laboratory conditions and real working conditions, and then, for validation purposes, analytical, heuristic evaluation with the involvement of usability experts were performed. During the development and the application of the method, the communication patterns of different teams were examined in terms of their relationship to software usability. In this paper we present the results of the second laboratory study and the heuristic evaluation in details. Overall, the method is able to explore team usability problems of different types of teams, making it a valuable element in existing methods for evaluating the usability of groupware.

Analisis User Experience pada Pengguna Aplikasi Dompet Digital Menggunakan Teori Jacob Nielsen

Digital wallets are one of the fastest-growing types of fintech. Digital wallets are just like real wallets. Users can make transactions either online or offline to be more effective and efficient. It is important for the development team to pay attention to the comfort and satisfaction of its users. The purpose of writing, to find out the rate obtained by each application and will be tested using the usability testing method. This is done to evaluate the usability of digital wallet applications based on the responses given by users after interacting with the product system. The research was conducted in the DKI Jakarta area using 3 digital wallets and involving 150 digital wallet respondents, including: 51 Shopee pay Respondents, 52 Go-pay Respondents and 47 DANA Respondents. The results of the tests that have been carried out, it can be concluded that the Shopee-pay Digital Wallet is in the Very Good category with a score of 4.35, the Go-pay and DANA applications are in the good category with the Go-Pay application score of 3.98 and DANA 4.16. This shows that the three applications have fulfilled the five aspects of usability testing: learnability, memorability, efficiency, errors and satisfaction.

BRIDGING THEORY AND PRACTICE: COMPARATIVE INSIGHTS INTO EVALUATING USER INTERFACES

Abstract Keywords Cite This Article as User interface (UI) evaluation is an important aspect of software development, ensuring that software or systems are efficient and user satisfactory. Among the various usability testing methods available, Cognitive Walkthrough Method, Heuristic Evaluation Method and Concurrent Usability Testing Method stand out as most popular approaches. This research paper presents a comparative analysis of these testing methods, emphasizing their strengths, flaws, and applicability in different situations. Through a detailed analysis of each testing method's procedural steps, and practical concerns, we aim to provide insights into their effectiveness and suitability for evaluating User Interface (UI) designs. Additionally, we discuss potential synergies and inadequacies when combining these testing methods in practice. Our results and findings contribute to a deeper understanding of UI evaluation techniques and provide guidance for selecting the most appropriate approach for testing team’s specific needs.

Evaluasi Usability Website Jurnal ABC Menggunakan Metode Usability Testing

Website-based public services are certainly an important part of supporting the success of a higher education institution, one of which is a journal website for the publication of scientific articles for lecturers and students. However, many of these website-based services are still difficult to use and do not function well. This research was conducted to evaluate whether the ABC Journal website has usability testing acceptability criteria and find out how much user acceptance the website has. The method used in this research is the Usability Testing method. Based on the results of data processing, the average value of the overall usability test to measure the use of the ABC Journal website for the Learnability aspect was 3.25, Efficiency was 3.30, Memorability was 3.20, Errors were 3.28 and Satisfaction was 3.23. These results show that the ABC Journal website is quite easy to use. Suggestions to the administrator of the ABC Journal Website, to increase the value of each aspect of usability so that it can be easy to use for its users.

Evaluation of Accounting Information System Using Usability Testing Method and System Usability Scale

The computer-based accounting information system IBS Core has been used by Renon Pekraman Village Credit Institution since 2016 to facilitate all transaction processes. This study aims to evaluate the usefulness of the IBS Core accounting information system, determine the effectiveness, efficiency, and user satisfaction with the IBS Core accounting information system, and identify areas that need improvement in the IBS Core accounting information system.The method used in this study is Usability Testing using performance measurement and retrospective think aloud (RTA) as well as the System Usability Scale (SUS). The results of the study show that the IBS Core system has a high effectiveness score of 92.50%. The average time required for participants to complete the task scenario is 68.9 seconds, and users feel that the available content is clear and consistent. In addition, the average System Usability Scale (SUS) score is 86.125, where the results were above the standard average SUS score. The IBS Core System Score was ranked B with the adjective ratings "Excellent” Next, the acceptability ranges are included in the "Acceptable" category, and finally the net promote score (NPS) is included in the "Promoter" category, showing that the use of the IBS Core system gets a very good assessment from its users. This shows that the IBS Core system is highly appreciated and considered very useful by its users.