Conduct Usability Testing Research Articles

Development of an energy management education program (“the PEP program”) for adults with end-stage renal disease

Introduction Fatigue is a highly common symptom of end-stage renal disease, interferes with occupational engagement, and is a top research priority of patients. The objective of this project was to develop an energy management education program that would meet the needs of people with end-stage renal disease and to conduct a preliminary evaluation of the acceptability and usability of the program. Methods We used the World Health Organization’s health education planning framework to guide the intervention development process. We systematically assessed the needs of people with end-stage renal disease related to energy management education, and transformed them into program objectives. Based on these findings, we designed a program that would (a) improve occupational engagement in people with end-stage renal disease; (b) be feasible; and (c) build on existing energy management education and health education literature. Finally, we conducted qualitative interviews about the program with four key informants and conducted usability testing with five target end-users. Findings: The Personal Energy Planning program combines an established problem-solving approach with three brief web modules on energy management. Preliminary testing suggested the program was acceptable to stakeholders and was usable by the target population. Conclusions Future research should explore the effects of the program on fatigue and occupational engagement in people with end-stage renal disease.

Feasibility and Acceptability of a Mobile Technology Intervention to Support Postabortion Care (The FACTS Study Phase II) After Surgical Abortion: User-Centered Design.

BackgroundHuman-centered design is a methodology that applies an iterative participatory process that engages the end-user for whom an innovation or intervention is designed for from start to end. There is general evidence to support the use of human-centered design for development of tools to affect health behavior, but specifically for family planning provision. This study is part two of a three-phase study that uses a user-centered design methodology which uses the findings from Phase I to design, develop, and test a digital health solution to support follow-up after an induced surgical abortion.ObjectiveThe objectives for this study were to: (1) develop a Web-based intervention based on preferences and experiences of women who underwent an abortion as measured in the formative phase of the Feasibility and Acceptability of a Mobile Technology Intervention to Support Postabortion Care Study; (2) conduct usability testing of the intervention to determine user-friendliness and appropriateness of the intervention; and (3) finalize a beta version of the Web-based intervention for pilot testing.MethodsThe study design was based on the “development-evaluation-implementation” process from the Medical Research Council Framework for Complex Medical Interventions. This study is in Phase II of III and is based on user-centered design methodology. Phase I findings demonstrated that women engage with technology to assist in clinical care and they preferred a comprehensive website with email or text notifications to support follow-up care. In Phase II we collaborated with family planning experts and key stakeholders to synthesize evidence from Phase I. With them and a development partner we built a prototype. Usability testing was completed with 9 participants using a validated System Usability Scale. This was then used to refine the intervention for Phase III pilot study. This study was approved by the local Ethics board.ResultsWe developed a comprehensive Web-based tool called myPostCare.ca, which includes: Post-Procedure Care, Emotional Well-Being Tool, Contraception Explorer, Sexual Health, Book an Appointment, and Other Resources. Additionally, over the course of a month after the procedure, automatic email notifications were sent to women as a form of virtual follow-up support, directing them to myPostCare.ca resources. The Web-based tool was refined based on usability testing results.ConclusionsThis study demonstrated that user-centered design is a useful methodology to build programs and interventions that are women-centered, specifically for abortion care.

Scope of Usability Tests in IS Development

Despite being a common, established concept in wide usage, usability tests can vary greatly in their goals, techniques, and results. A usability test that one purchases and performs for a specific software product may result in either minor user interface improvements or radical U-turns in development. Researchers have discussed such variation as a problem that concerns testing method’s scientific reliability and validity. In practice, what “kind of data” one can expect to obtain from the selected method has more importance than whether one always obtains the same data. This expectation about information content or “scope” has importance for those who select and conduct usability tests for a specific purpose. However, researchers rarely explicitly state or even discuss scope: too often they adopt the premise that, because a usability test involves users, it brings the (necessary) user-centeredness to the design (i.e., takes socio- technical fundamentals as inherently given). We reviewed the literature on testing practices and analytical consideration and searched for the scope of a usability test that could deliberately approach the socio-technical tradition and equally develop both the system and the user organization. A case example represents a possible realization of the extended scope of usability test.

Beyond Reality-Extending a Presentation Trainer with an Immersive VR Module.

The development of multimodal sensor-based applications designed to support learners with the improvement of their skills is expensive since most of these applications are tailor-made and built from scratch. In this paper, we show how the Presentation Trainer (PT), a multimodal sensor-based application designed to support the development of public speaking skills, can be modularly extended with a Virtual Reality real-time feedback module (VR module), which makes usage of the PT more immersive and comprehensive. The described study consists of a formative evaluation and has two main objectives. Firstly, a technical objective is concerned with the feasibility of extending the PT with an immersive VR Module. Secondly, a user experience objective focuses on the level of satisfaction of interacting with the VR extended PT. To study these objectives, we conducted user tests with 20 participants. Results from our test show the feasibility of modularly extending existing multimodal sensor-based applications, and in terms of learning and user experience, results indicate a positive attitude of the participants towards using the application (PT+VR module).

Responsive Asthma Care for Teens (ReACT): development protocol for an adaptive mobile health intervention for adolescents with asthma

IntroductionAsthma is a leading cause of youth morbidity in the USA, affecting >8% of youth. Adherence to inhaled corticosteroids (ICS) can prevent asthma-related morbidity; however, the typical adolescent with asthma...

SwapMyMood: An iOS App for Problem-Solving and Emotion Regulation after TBI

Conduct usability testing of a mobile app that incorporates evidence-based meta-cognitive strategies for emotion regulation (the emotion cycle) and problem-solving (SWAPS) used in the Short-Term Executive Plus (STEP) cognitive rehabilitation model.

Badania użyteczności systemów w instytucjach publicznych na przykładzie Ośrodka Przetwarzania Informacji – Państwowego Instytutu Badawczego

The article describes the specifics of usability research in public institutions on the example of work at the National Information Processing Institute. It describes the challenges faced by system creators, designers, and researchers. It presents methods of preparing and conducting usability tests (preparation, execution and further steps after researches). Characterization of systems created in the institute: POL-on, JSA shows specifics of working at the public systems and shares the insights from these researches. Based on the gathered information, the article proposes changes to improve the experience of systems’ users. It presents good practices that the creators may follow during the designing process, such as: naming, icons, charts, cohesion, searching, text editing and information architecture.

Determining fidelity of mixed prototypes: Effect of media and physical interaction

Mixed prototyping, combining virtual and physical prototypes, is an emerging method used to aid in usability testing. This study aims to determine when to use a mixed prototype and how to choose its fidelity to validate the usability testing results and reduce the prototyping cost. A 2×2 between-subject experiment was designed to investigate the effects of the media (head-mounted display versus computer monitor) and physical interaction (using a tangible mock-up or not) on the usability evaluation results and other subjective measures. The experiment results showed that, when aesthetic and functional features are controlled, the non-functional mock-up facilitates users in finding problems regarding physical interaction and ergonomics. Media with high immersion positively influenced the users’ subjective ratings. Based on the findings of this study, guidelines on how to choose the fidelity of the prototype during different stages of a usability test are suggested to help product developers find a cost-efficient way to conduct usability tests.

A Proposed Web Based Real Time Brain Computer Interface (BCI) System for Usability Testing

Hallway testing, Remote Usability testing, Expert review, Automated expert review and A/B testing are the methods commonly used for Usability testing. However, there is no reliable system that integrates Brain Computer Interface (BCI) into the testing process with focus given towards user emotion analysis using electroencephalography (EEG) signals. This paper proposes a system that would be able to identify user emotions while they are conducting usability tests and the results would be able to increase the accuracy of the usability test. In the proposed system the results of the usability test would be displayed in real time on a dashboard and a summary report can be generated for distribution.

A Proposed Web Based Real Time Brain Computer Interface (BCI) System for Usability Testing

Hallway testing, Remote Usability testing, Expert review, Automated expert review and A/B testing are the methods commonly used for Usability testing. However, there is no reliable system that integrates Brain Computer Interface (BCI) into the testing process with focus given towards user emotion analysis using electroencephalography (EEG) signals. This paper proposes a system that would be able to identify user emotions while they are conducting usability tests and the results would be able to increase the accuracy of the usability test. In the proposed system the results of the usability test would be displayed in real time on a dashboard and a summary report can be generated for distribution.

Usability Testing of a Mobile Clinical Decision Support App for Urinary Tract Infection Diagnosis in Nursing Homes

The aim of the current study was to conduct usability testing of a mobile clinical decision support (CDS) prototype designed for urinary tract infection (UTI) assessment by nurses in nursing homes (NHs). Usability of the UTIDecide smartphone application (app) was evaluated using cognitive walk-through and think-aloud protocol sessions with nurses (n = 6) at two NH sites. This evaluation was followed by unsupervised field tests lasting ≥1 week with nurses at one site (n = 4) and posttest interviews and administration of the System Usability Scale (SUS). Cognitive walk-through/think-aloud sessions yielded interface design recommendations that were implemented prior to field tests. All test sessions resulted in highly positive perceived usability and usefulness from participants. Average SUS score was 92.5 (n = 3), which equates to an "A" grade for usability. Design recommendations identified for future app versions are: (a) integration of the mobile CDS app with organizational information systems; and (b) expanded features to support assessment of other conditions. [Journal of Gerontological Nursing, 45(7), 11-17.].

3080 Ensuring Quality in Investigator-Initiated Clinical Trials through Monitoring Concepts Training

OBJECTIVES/SPECIFIC AIMS: Because clinical trial results are instrumental in the approval of a new drug or changes to the practice of medicine, ensuring the accuracy and validity of collected data is critical in the clinical trial process. This function, routinely carried out by clinical trial monitors in industry-sponsored trials, is often lacking in investigator-initiated trials (IITs) conducted in academia. To address this challenge, we have developed a self-study module that can be used to cross-train academic researchers in essential concepts and practical approaches to monitoring. Furthermore, we are applying a framework drawn from implementation science in the development and launch of this initiative. This framework, as used in other educational programs, is employed here to close the gap between initiative and practice, thereby effectively disseminating this training would improve the quality of clinical trials in academia. METHODS/STUDY POPULATION: This research project applied exploration, installation and implementation stages of the implementation science process by 1) exploring the need for a new initiative, 2) disseminating results, 3) engaging stakeholders, 4) creating standard operating procedures (SOPs) for installation and implementation, 5) studying user satisfaction and effectiveness, 6) addressing feedback and 7) conducting implementation. RESULTS/ANTICIPATED RESULTS: From literature review and internet searches we determined that although numerous GCP training resources exist, most are too broad and lack the practical approaches to meet the complex requirements of monitoring. Moreover, most of the offerings identified are costly or inaccessible. With only about 65% of IITs reported as being monitored (Figures 1 and 2), it appears that there is a clear need for training tools that are easily available to a broader audience. And because monitoring skills are substantially different from those associated with research coordination, it is not surprising that research professionals believed that they would need additional training to become proficient. To address this need, we began developing a monitoring module. We engaged key stakeholders from academia and industry to gain insights into their needs. The results indicated that although our training module was effective, supplementary information on the fundamentals of clinical trials should be included for those new to the field. After incorporating suggested changes and completing the module, we conducted user testing to determine if our module is ready to be broadly disseminated (Figures 3 and 4). Following positive feedback from the group, we are currently in the process of disseminating our module and studying its impact. DISCUSSION/SIGNIFICANCE OF IMPACT: IITs are instrumental in translating academic research into product development. Deficiencies in the quality control of these trials can lead to inadequacies in data accuracy and validity that could lead to significant delays in bringing innovative therapies to patients. Recent NIH policies require data and safety monitoring for all of the trials it supports. The latest addendum to ICH GCP, E6(R2), discusses a need for quality management across the clinical trial lifecycle. As we continue to disseminate and share information during the development of our self-study monitoring module, we are engaging key stakeholders from academia, government, and private institutions to understand and address quality challenges in conducting clinical trials. Finally, this research informs dissemination and implementation research, specifically for creating training for academic research professionals.

Usability and usefulness of internet mapping platforms in participatory spatial planning

Internet mapping platforms are widely used in participatory planning. They carry the promise of accessibility, a wider audience, and more precise spatial data. However, they are critiqued for issues of trust, digital exclusion, and low usability. While the first two problems can be dependent on the local context, the last one is connected with the introduction of map editing tools. They change browser experience and require accessing spatial knowledge, which is a cognitively demanding task, especially for older people. This can negatively influence both quality and quantity of the data being produced. In this paper, we ask questions about the perception of online forms of participation, the accuracy and reliability of the data being collected in participatory mapping, differences between participants age groups, and design considerations for creating successful and inclusive participatory mapping tools? We conducted usability testing experiment with 30 participants. They were presented with various map editing tasks that were to be completed using a mock internet participatory mapping platform that was designed to mimic real-life applications. The sessions were recorded using eye-tracking equipment.We have observed a considerable enthusiasm for using and interacting with web mapping tools, but this was not reflected in the quality of the data which was very low overall. Known map features were added in wrong places even in simpler tasks. Although there were considerable differences between the oldest and youngest people in the map editing behavior, the quality and quantity of the data were similar. We conclude that mapping platforms can be useful for participatory planning and age of the participants is not a barrier for their inclusion when the interface is properly designed. But in the same time care is needed to provide robust quality control. We also propose a research agenda composed from three avenues of research: data quality, usability and user experience and utility of the map.

User Testing: Gathering Data from First-Year Medical Students as They Interact with the EBSCO Discovery Service (EDS)

Researchers from the Mercer University School of Medicine and EBSCO Information Services formed a partnership to conduct user testing on the library’s EBSCO Discovery Service (EDS) interface. The goal of the study was to determine if first-year medical students encountered difficulty when asked to retrieve full-text, eBook, peer-reviewed, and print items relevant to their current research. Participants experienced some confusion when formulating search terms and selecting and applying limiters to refine searches.

Developing Usable Academic Library Website for International Students through Usability Testing

A usable website is a key to ensuring user satisfaction whiles an unusable website brings about frustration and user dissatisfaction. Usability testing has been identified as a great way to check how usable products are to users. This is however ignored by many designers during the designing stage of products such as websites. This in the long run makes the user experience difficulties, errors, and frustration because the product was not created with the user needs in mind. Usability testing is a great way to explore the needs of users and also identify problems and challenges that users encounter while using a product. Involving users in usability testing helps to produce user-centered websites. This study therefore is an attempt to examine the Jiangsu University English Library Website which was designed for international students. Thus the study seeks to identify the needs of the international students as well as problems the users face whiles using the academic library website by involving them in the usability test. A user evaluation usability test with the Morae software was used in this study to analyze data. The study involves twenty international student participants of whom ten were from Asian Countries outside China and the other ten were from African countries and compared usability issues encountered by the groups when using the JUELW. According to the rule of the thumb 5 users are enough to help explore the usability issues. This study therefore serves as a guide for various libraries to use while conducting usability tests for their users and also makes recommendation for libraries to consider when designing library websites for their users especially international student.

Penerapan Metode User Centered Design Untuk Mengembangkan E-Learning Universitas Jember Berbasis Mobile

Aspects of usability or usability in the user's point of view when using and utilizing a product. The use of an inappropriate method in developing the design will increase the number of requests to improve the product after it is finished. Inappropriate method of selecting methods for design development Usually occurs in applications based on Graphical User Interface (GUI), e-commerce websites, cellular services, and also technology in the field of education called e-learning. E-learning is one technology that develops a distance learning system that provides a variety of solutions where the Internet is the medium. Supported by the development of technology supported by smartphones can also be used for the learning process. With the existence of smartphones, students will more easily access learning materials, and teachers will also be easier to share the material to be received. To support the effort to obtain the level of user satisfaction without leaving the specific purpose of making the application itself a user centered design (UCD) method can be applied. UCD is a method that helps users. With the application of this method users will be satisfied and comfortable when using with the system. Based on the concepts of the UCD method and the problems that occur in e-learning especially the University of Jember E-Learning application, it can be seen that the UCD method is suitable to be applied in the development of mobile-based Universitas Jember e-learning applications with the aim of making it easier students and lecturers in the learning process at the University of Jember. This research was conducted in 3 stages, namely data collection, analysis, and system development. The data collection phase consists of literature studies, and questionnaires. The analysis phase is done by processing the data from the questionnaire and conducting usability testing. The development phase is done by building a mobile-based e-learning application. The results of this study are the University of Jember M-Learning application.

Introducing an Electronic Identity: The Co-design Approach in the Canton of Schaffhausen

Because of the Schaffhausen population’s needs regarding more flexibility of authorities, an augmented importance of online services and increases in (cyber) security awareness, the e-government committee (eGov-Kernteam) of the Canton of Schaffhausen decided to pilot the electronic identity Schaffhauser eID+ to ultimately allow for efficient and effective Smart Government processes.* The aim of this practice article is to show how the Canton of Schaffhausen adopted a co-design approach to introduce its electronic identity and to set out key findings regarding the introduction of a technology-based public service innovation. Based on the strategic approach of a ‘testing laboratory’, co-design methodology was used through conducting usability tests and semi-structured interviews with selected individuals. The optimization suggestions collected were subsequently evaluated and reintegrated. Furthermore, the collected data were used to analyze the acceptance of the Schaffhauser eID+. This piloting procedure in cooperation with citizens unites several advantages: modest budgets, rapid and more targeted product changes and, in the sense of co-design, positive spillovers with regard to acceptance and legitimacy. In reference to Smart Government and to ensure ongoing civic participation, the co-design approach should also be considered in the further development of any E-ID or Smart Government project.

Not Just a Pretty Picture: The U.S. Cancer Statistics Data Visualization Tool

Background: The United States Centers for Disease Prevention and Control (CDC) National Program of Cancer Registries (NPCR) works to disseminate cancer surveillance data to multiple audiences in accessible, discoverable, and usable formats. To this end, CDC released the official federal cancer statistics, U.S. Cancer Statistics (USCS), in a data visualization tool available at www.cdc.gov/cancer/dataviz . We made further enhancements to the online tool since its initial release in 2017. Aim: We describe the process and enhancements made to the USCS data visualization tool's content, graphical displays, and sharing capabilities. Methods: CDC partnered with Agency for Toxic Substances and Disease Registry's Geospatial Research, Analysis, and Services Program to further work initiated by a group of cancer registrars, program planners, epidemiologists, computer programmers, and communication specialists to improve the visual presentation of USCS cancer incidence and mortality data. We conducted usability testing and implemented changes to the Web site's layout and added content, including county-level data, survival data, and prevalence estimates. Results: New features include county statistics, survival, prevalence, Puerto Rico data, and tobacco-, alcohol-, and obesity-related cancers data displays. The tool was also enhanced to better display on mobile devices. Data displays on national and state incidence, mortality and trends are available as maps and bar charts with interpretive text when users scroll over each graphic. Users can customize displays of overall and cancer-specific statistics, download data tables, and share each page via social media. Conclusion: Surveillance data are fundamental to measure progress and target action. CDC's interactive USCS data visualization tool is designed to make cancer data more accessible and usable to multiple users, including the general public, media, policy makers and planners. We will continue to improve the tool's accessibility and usefulness to facilitate the interpretation and sharing of cancer data.

Moderating Styles: Understanding Diverse Techniques for Investigating the Root Cause of Use Events

This article explores how a moderator’s style shapes the techniques they use for conducting usability testing on medical devices. Different moderating styles offer distinct routes to arrive at analogous root-cause data when the moderator is appropriately trained. We interviewed three moderators from a human factors medical device consulting company about how they approach investigating the root cause of use events. A typology of three styles emerged from the interviews: Direct Moderator, Indirect Moderator, and Social Moderator. No style was inherently superior. Each moderator employs a distinct toolkit to gain insights about users’ interactions with devices. These data provide product developers with either recommendations for device improvements or with documentation for human factors FDA validation.Moderators may gain from learning about alternate moderating styles to enhance their mental flexibility during testing sessions. Knowledge of various styles also encourages a more diverse set of practitioners to enter the human factors field.

Designing Social Question-and-Answering Interaction Using Goal-Directed Design Method

A social question-and-answering (social Q&A) site is an online community where users can ask and answer questions to share their knowledge and all data are publicly accessible. The platform has caught its users’ attention and given them a new way in getting information from other users. However, the growth of social Q&A sites is still followed by some problems faced by its users. This study aims to develop an interaction design of social Q&A using goal-directed design (GDD) method. Compared to User-Centered Design method, the GDD has more specific process. The process accomplished in the development includes four phases: research, modeling, requirement definition, and framework definition. The research phase was done by conducting literature review and potential user interview. The modeling phase was carried out to explore behavior patterns, and combine them into user model. The requirement definition was conducted to determine product functionality and design. In this phase, user scenarios that describe how a persona uses the product were developed. The framework definition phase focused on an interaction framework which was conducted through six steps. The series of phases resulted in personas, scenarios, requirements, and wireframes of the product. Social Q&A product developed in the study is one that is specialized for asking and sharing information about studying abroad. The design was developed in the form of product wireframes and key path scenarios along with storyboards that describe interaction in the product. Key path scenarios were constructed based on tasks personas could perform in the product. Further works include developing visual design framework and conducting usability testing of the design to potential users.