Identify Design Issues Research Articles

End user experience of a widely used artificial intelligence based sepsis system.

Research on the Epic Sepsis System (ESS) has predominantly focused on technical accuracy, neglecting the user experience of healthcare professionals. Understanding these experiences is crucial for the design of Artificial Intelligence (AI) systems in clinical settings. This study aims to explore the socio-technical dynamics affecting ESS adoption and use, based on user perceptions and experiences. Resident doctors and nurses with recent ESS interaction were interviewed using purposive sampling until data saturation. A content analysis was conducted using Dedoose software, with codes generated from Sittig and Singh's and Salwei and Carayon's frameworks, supplemented by inductive coding for emerging themes. Interviews with 10 healthcare providers revealed mixed but generally positive or neutral perceptions of the ESS. Key discussion points included its workflow integration and usability. Findings were organized into 2 main domains: workflow fit, and usability and utility, highlighting the system's seamless electronic health record integration and identifying design gaps. This study offers insights into clinicians' experiences with the ESS, emphasizing the socio-technical factors that influence its adoption and effective use. The positive reception was tempered by identified design issues, with clinician perceptions varying by their professional experience and frequency of ESS interaction. The findings highlight the need for ongoing ESS refinement, emphasizing a balance between technological advancement and clinical practicality. This research contributes to the understanding of AI system adoption in healthcare, suggesting improvements for future clinical AI tools.

ProcessGallery: Contrasting Early and Late Iterations for Design Principle Learning

Traditional design galleries enable users to search for examples based on surface attributes (e.g., color or style), and largely obscure underlying principles (e.g., hierarchy or readability). We conducted three studies to explore how galleries could be constructed to help novices learn key design principles. Study 1 revealed that novices gain perspective by observing how designs evolve throughout a process. Study 2 found that novices are better at identifying design issues when viewing iterations that show improvements for just one principle at a time, rather than multiple. Building on these insights, we created ProcessGallery, a tool that enables users to browse contrasting pairs of early-and-late iterations of designs that highlight key improvements organized by design principles. In Study 3, a within-subjects experiment, sixteen participants iterated on a seed design after viewing examples in ProcessGallery versus a traditional gallery. Using ProcessGallery, participants found more appropriate examples, assessed designs better, and preferred ProcessGallery for learning compared to a traditional gallery.

Automated Gameplay Testing and Validation With Curiosity-Conditioned Proximal Trajectories

This paper proposes a novel deep reinforcement learning algorithm to perform automated analysis and detection of gameplay issues in complex 3D navigation environments. The Curiosity-Conditioned Proximal Trajectories (CCPT) method combines curiosity and imitation learning to train agents that methodically explore in the proximity of known trajectories derived from expert demonstrations. We show how our new algorithm can explore complex environments, discovering gameplay issues and design oversights in the process, and recognize and highlight them directly to game designers. We also propose a visual analytics interface to aid interpretation of results from the method. This interface transforms information from complex models into interpretable and interactive visual forms. We further demonstrate the effectiveness of the algorithm in a novel 3D navigation environment which reflects the complexity of modern video games. Our results show a higher level of coverage and bug discovery than baseline methods, demonstrating that our method can be a useful tool for game designers to automatically identify design issues. Moreover, our experiments show that the visual explanations provided by the analytics interface result in a significant increase in user trust and acceptance of automated playtesting and increased confidence in the use of machine learning techniques for video game development.

Mixed methods usability evaluation of an assistive wearable robotic hand orthosis for people with spinal cord injury

BackgroundRobotic hand orthoses (RHO) aim to provide grasp assistance for people with sensorimotor hand impairment during daily tasks. Many of such devices have been shown to bring a functional benefit to the user. However, assessing functional benefit is not sufficient to evaluate the usability of such technologies for daily life application. A comprehensive and structured evaluation of device usability not only focusing on effectiveness but also efficiency and satisfaction is required, yet often falls short in existing literature. Mixed methods evaluations, i.e., assessing a combination of quantitative and qualitative measures, allow to obtain a more holistic picture of all relevant aspects of device usability. Considering these aspects already in early development stages allows to identify design issues and generate generalizable benchmarks for future developments.MethodsWe evaluated the short-term usability of the RELab tenoexo, a RHO for hand function assistance, in 15 users with tetraplegia after a spinal cord injury through a comprehensive mixed methods approach. We collected quantitative data using the Action Research Arm Test (ARAT), the System Usability Scale (SUS), and timed tasks such as the donning process. In addition, qualitative data were collected through semi-structured interviews and user observations, and analyzed with a thematic analysis to enhance the usability evaluation. All insights were attributed and discussed in relation to specifically defined usability attributes such as comfort, ease of use, functional benefit, and safety.ResultsThe RELab tenoexo provided an immediate functional benefit to the users, resulting in a mean improvement of the ARAT score by 5.8 points and peaking at 15 points improvement for one user (clinically important difference: 5.7 points). The mean SUS rating of 60.6 represents an adequate usability, however, indicating that especially the RHO donning (average task time = 295 s) was perceived as too long and cumbersome. The participants were generally very satisfied with the ergonomics (size, dimensions, fit) of the RHO. Enhancing the ease of use, specifically in donning, increasing the provided grasping force, as well as the availability of tailoring options and customization were identified as main improvement areas to promote RHO usability.ConclusionThe short-term usability of the RELab tenoexo was thoroughly evaluated with a mixed methods approach, which generated valuable data to improve the RHO in future iterations. In addition, learnings that might be transferable to the evaluation and design of other RHO were generated, which have the potential to increase the daily life applicability and acceptance of similar technologies.

Participatory budgeting and the perception of collective empowerment: institutional design and limited political interference

Participatory budgeting gains momentum around the world, and increasing evidence provides mixed results about its effects. Under these circumstances, it is unclear if citizens consider it a source of empowerment and an avenue for effective decision-making in the life of their local community. We know very little about how participants in participatory budgeting perceive the collective empowerment. This article seeks to identify the factors that shape these perceptions about the empowerment potential of participatory budgeting. It focuses on the critical case of Cluj-Napoca and uses 25 semi-structured interviews conducted in October–November 2020 with three categories of participants. Our findings indicate that participants acknowledge the potential for collective empowerment and praise the limited political involvement but identify design issues and resource allocation as weakening the empowerment potential.

Discovering experimental design: An interactive teaching exercise using Fisher's tea‐tasting experiment

AbstractAn appreciation of experimental design is an important aspect of introductory statistics teaching in a wide range of applied disciplines, including medical statistics. Understanding the impact of design decisions on the choice of analysis method and subsequent interpretation of results can help to embed the importance of statistical thinking in the experimental process. I discuss an interactive exercise, based on R.A. Fisher's celebrated “Lady Tasting Tea” experiment, that is intended to raise awareness of design issues as part of an undergraduate statistics module. The exercise used a discovery approach, with students encouraged to identify design issues and agree on solutions themselves via small group discussion, with only low‐level prompting from the instructor. The value of this teaching style and possible extensions of the tea‐tasting experiment to other related topics suitable for more widespread use are also discussed.

MUP: Simplifying Secure Over-The-Air Update with MQTT for Constrained IoT Devices

Message Queuing Telemetry Transport (MQTT) is one of the dominating protocols for edge- and cloud-based Internet of Things (IoT) solutions. When a security vulnerability of an IoT device is known, it has to be fixed as soon as possible. This requires a firmware update procedure. In this paper, we propose a secure update protocol for MQTT-connected devices which ensures the freshness of the firmware, authenticates the new firmware and considers constrained devices. We show that the update protocol is easy to integrate in an MQTT-based IoT network using a semantic approach. The feasibility of our approach is demonstrated by a detailed performance analysis of our prototype implementation on a IoT device with 32 kB RAM. Thereby, we identify design issues in MQTT 5 which can help to improve the support of constrained devices.

Applying citizen science to gene, drug and disease relationship extraction from biomedical abstracts.

Biomedical literature is growing at a rate that outpaces our ability to harness the knowledge contained therein. To mine valuable inferences from the large volume of literature, many researchers use information extraction algorithms to harvest information in biomedical texts. Information extraction is usually accomplished via a combination of manual expert curation and computational methods. Advances in computational methods usually depend on the time-consuming generation of gold standards by a limited number of expert curators. Citizen science is public participation in scientific research. We previously found that citizen scientists are willing and capable of performing named entity recognition of disease mentions in biomedical abstracts, but did not know if this was true with relationship extraction (RE). In this article, we introduce the Relationship Extraction Module of the web-based application Mark2Cure (M2C) and demonstrate that citizen scientists can perform RE. We confirm the importance of accurate named entity recognition on user performance of RE and identify design issues that impacted data quality. We find that the data generated by citizen scientists can be used to identify relationship types not currently available in the M2C Relationship Extraction Module. We compare the citizen science-generated data with algorithm-mined data and identify ways in which the two approaches may complement one another. We also discuss opportunities for future improvement of this system, as well as the potential synergies between citizen science, manual biocuration and natural language processing. Mark2Cure platform: https://mark2cure.org; Mark2Cure source code: https://github.com/sulab/mark2cure; and data and analysis code for this article: https://github.com/gtsueng/M2C_rel_nb. Supplementary data are available at Bioinformatics online.

Mobile Health Apps: Improving Usability for Older Adult Users

With smartphone use among older populations on the rise, older adults have increased access to health-focused mobile apps. Despite their potential benefits for managing health, currently no guidelines exist for designing these apps specifically for older adult users. We evaluated the usability of one medication management app and two congestive heart failure management apps using cognitive walkthroughs, heuristic analysis, and user testing. We identified design issues that may affect usability for older users, including poor navigation, small button sizes, and inadequate data visualizations. We provide guidelines for developers of health apps to facilitate use by older adults.

The heuristic version of Cognitive Work Analysis: A first application to medical emergency situations

Cognitive Work Analysis is an original method that seeks to describe work systems made up of nested sets of constraints, from ecological constraints imposed by the work domain to cognitive constraints. This top-down approach starts with a work domain model in order to analyze and specify contexts of activity. To complement this method, we propose a bottom-up version of Cognitive Work Analysis focusing on contexts of activity and depicting how operators adapt to the ecological constraints. Based on Rasmussen's Dynamic Safety Model, the ecological constraints involved are those bounding the workspace in which operators dynamically navigate with control loops, strategies, work organization, and competencies. This analysis relies on the simulation of specific contexts of activity. A first illustrative application of this framework to a simulated medical emergency situation with a team of nurses and nursing aids shows that this framework can help identify design issues.

An Industry Case Study: Investigating Early Design Decision Making in Virtual Reality

The research presented here describes an industry case study of the use of immersive virtual reality (VR) as a general design tool with a focus on the decision making process. A group of design and manufacturing engineers, who were involved in an active new product development project, were invited to participate in three design reviews in an immersive environment. Observations, interviews, and focus groups were conducted to evaluate the effect of using this interface on decision making in early product design. Because the team members were actively engaged in a current product design task, they were motivated to use the immersive technology to address specific challenges they needed to solve to move forward with detailed product design. This case study takes the approach of asking not only what can users do from a technology standpoint but also how their actions in the virtual environment influence decision making. The results clearly show that the team identified design issues and potential solutions that were not identified or verified using traditional computer tools. The design changes that were the outcome of the experience were implemented in the final product design. Another result was that software familiarity played a significant role in the comfort level and subsequent effectiveness of the team discussions. Finally, participants commented on how the immersive VR environment encouraged an increased sense of team engagement that led to better discussions and fuller participation of the team members in the decision process.

Characterizing mobile *-box applications

With the increasing use of multiple electronic devices including tablets, PCs, and mobile devices, Personal Cloud Storage (PCS) services, such as Dropbox and Box, have gained huge popularity. Recent research has used the PC clients of a few PCS services to study the network architectures and performance of these services. The mobile clients deserve a further study because the study of PC clients does not necessarily represent the system and network demand with mobile clients. In this paper, we conduct the first systematic investigation on six popular PCS services to reveal their internals and measure their performance. By dissecting their protocols and conducting cross-layer examinations, we obtain interesting observations, identify design issues, and suggest solutions to remedy these issues. Moreover, we propose an efficient method to measure the response latency of PCS servers by exploiting their open APIs.

Usability Evaluation Survey for Identifying Design Issues in Civil Flight Deck

Ergonomics assessment for cockpit in civil aircraft is important as the pilots spend most of their time during flight on the seating posture imposed by its design. The improper seat design can cause discomfort and pain, which will disturb the pilot's concentration in flight. From a conducted survey, it is found that there are some issues regarding the current cockpit design. This study aims to highlight potential mismatches between the current cockpit design and the ergonomic design recommendations for anthropometric dimensions and seat design, which could be the roots of the problems faced by the pilots in the cockpit.

Business Model for Mobile Payment in China

Despite the predicted success of mobile payment, the market remains immature in most countries. Major concerns are the relationship between push and pull technologies, and the role of platforms, service innovation, power and control in ecosystems. As the first step in their mixed-method research approach, using Analytic Hierarchy Process (AHP) as a research approach and the STOF business model as a research framework, the authors aim to identify design issues for m-payment services from mobile network operators' perspectives. Furthermore, the authors compare insights from semi-structured interviews with experts in the field to the empirical findings, to assess how the actual business model of Chinese m-payment services can be improved based on design issues derived from the business model. The results show that components such as building customer trust on payment services, innovative payment experience, and extend market to new segments, guarantee security and privacy issues, user profile management, and hardware problems involving existing infrastructure, customer/merchant relationship, platform interoperability, and cost saving on fraud detection need to be improved to enhance the potential of m-payment, supported by a viable and sustainable business model. There is also a role for policy and regulation to be played.

Identifying Latent Design Issues in Mobile Products to Prevent Patient Harm

This paper describes a rigorous yet practical process for evaluating mobile health care products (native and webbased software applications) to identify design issues that present a risk to patients. The process uses a model-based approach to application analysis, combining techniques from human factors, human-computer interaction, safety engineering and patient safety. Examples from mobile products in development are used to illustrate how the process facilitates detection of common design issues. Lessons learned from developing and applying the process also are discussed.

Push Notification Mechanisms for Pervasive Smartphone Applications

Smartphones have emerged as ubiquitous devices that are being used to host a rich array of pervasive apps in diverse areas, including healthcare, gaming, and social networking. A key requirement of many of these apps is that they keep their mobile users up-to-date as changes occur in some subject of interest. One way of providing timely updates to smartphone apps is to use push notification technology. In this article, the authors identify design issues for push notification systems and reveal underlying concepts for the notification channels that bind mobile devices to push services. They describe five service offerings and report on the performance of their channels based on an empirical study.

Software architecture review by association

During the process of software design, software architects have their reasons to choose certain software components to address particular software requirements and constraints. However, existing software architecture review techniques often rely on the design reviewers’ knowledge and experience, and perhaps using some checklists, to identify design gaps and issues, without questioning the reasoning behind the decisions made by the architects. In this paper, we approach design reviews from a design reasoning perspective. We propose to use an association-based review procedure to identify design issues by first associating all the relevant design concerns, problems and solutions systematically; and then verifying if the causal relationships between these design elements are valid. Using this procedure, we discovered new design issues in all three industrial cases, despite their internal architecture reviews and one of the three systems being operational. With the newly found design issues, we derive eight general design reasoning failure scenarios.

Development and evaluation of a novel taekwondo chest protector to improve mobility when performing axe kicks.

The axe kick, in Olympic style taekwondo, has been identified as the most popular scoring technique aimed to the head during full contact competition. The first purpose of this study was to identify and investigate design issues with the current World Taekwondo Federation approved chest protector. A secondary purpose was to develop a novel chest protector addressing the identified design issues and to conduct a biomechanical analysis. Fifteen male elite Taekwondo players were selected to perform three different styles of the axe kick, i.e., front, in-out, and out-in axe kick five times each for a total of 45 kicks. Two-way repeated measures ANOVA showed significant differences between the novel and existing chest protector conditions for vertical height of the toe, downward kicking foot speed, hip flexion angle and ipsilateral shoulder flexion extension range of motion (ROM) (p < 0.05). There were no significant differences between the control condition (no chest protector) and the novel chest protector condition for these variables (p > 0.05). These results indicate that the novel chest protector interferes less with both the lower and upper limbs during the performance of the axe kick and provides a more natural, free-moving alternative to the current equipment used.

Evaluation of user Interface Design for Leaning Management System (LMS): Investigating Student's Eye Tracking Pattern and Experiences

E-learning is an innovation technology with the aim to provide a strategy to improve the quality of teaching and learning. The establishment of LMS is to improve the teaching and learning environments and to have better learning delivering through electronic device. The interface design of LMS plays an important role to deliver the learning process to both learners and educators. The usability of LMS is an important criterion for providing an effective and easy learning environment. Thus, this study aimed to evaluate user interface design of LMS by analyzing student's eye tracking pattern though the gaze plot and heat map. The interface of LMS was divided into three area; top, left and content. Student's area of interest was identified using the eye tracing pattern. Students eye movement was studied using Tobii Series Eye Tracker Model T60/T12 and ManGold software suite to capture their eye tracking pattern when using the LMS to complete their task. The analysis from the student's eye tracking pattern indicated some interface design issues in LMS. Based on this finding, we discuss the user interface design guidelines applied in LMS and suggested some improvement for identified design issues.

Utilization of two web-based continuing education courses evaluated by Markov chain model

To evaluate the web structure of two web-based continuing education courses, identify problems and assess the effects of web site modifications. Markov chain models were built from 2008 web usage data to evaluate the courses' web structure and navigation patterns. The web site was then modified to resolve identified design issues and the improvement in user activity over the subsequent 12 months was quantitatively evaluated. Web navigation paths were collected between 2008 and 2010. The probability of navigating from one web page to another was analyzed. The continuing education courses' sequential structure design was clearly reflected in the resulting actual web usage models, and none of the skip transitions provided was heavily used. The web navigation patterns of the two different continuing education courses were similar. Two possible design flaws were identified and fixed in only one of the two courses. Over the following 12 months, the drop-out rate in the modified course significantly decreased from 41% to 35%, but remained unchanged in the unmodified course. The web improvement effects were further verified via a second-order Markov chain model. The results imply that differences in web content have less impact than web structure design on how learners navigate through continuing education courses. Evaluation of user navigation can help identify web design flaws and guide modifications. This study showed that Markov chain models provide a valuable tool to evaluate web-based education courses. Both the results and techniques in this study would be very useful for public health education and research specialists.