End-user Usability Research Articles

Liver Disease Detection using Machine Learning

Abstract: Liver disease is a serious worldwide health issue, and prompt diagnosis and treatment are essential for successful outcomes. Traditional diagnostic techniques, however, may be expensive and time-consuming and can require intrusive procedures. In this work, we suggest a machine learning-based method for liver disease identification that makes use of the Support Vector Machine (SVM) and Random Forest Decision Tree algorithms. Our approach uses a large dataset with pertinent clinical characteristics including biochemical signs and patient demographics to categorize people into liver disease-positive or - negative groups. Furthermore, we incorporate an intuitive UI with Streamlit, improving accessibility and usability for end users and healthcare providers alike. By means of meticulous testing and assessment, we exhibit the efficiency and functionality of our suggested framework. In terms of precision, specificity, and sensitivity, our model performs admirably, demonstrating its promise as a trustworthy instrument for the identification of liver illness. Moreover, the incorporation of Streamlit improves the system's practicality and usability, making it easier to install and use in actual healthcare environments.

Scenario planning for climate adaptation and management: a high-level synthesis and standardization of methodology

Scenario planning is a tool used to explore a set of plausible futures shaped by specific trajectories. When applied in participatory contexts, it is known as participatory scenario planning (PSP), which has grown in its usage for planning, policy, and decision-making within the context of climate change. There has been no high-level synthesis of systematic reviews covering the overall state and direction of PSP for climate adaptation and management. We draw from four systematic reviews on PSP published between 2015 and 2021 to substantiate the credibility of the process and identify a set of standard practices to make PSP a more accessible and usable tool for not only researchers, but policymakers, practitioners, and other end users who may benefit from PSP. We summarize and synthesize the range of PSP processes and characteristics, highlighting four common trends that provoke additional inquiry: PSP’s contribution to social learning and bias, the varying use of quantitative information in scenario development, issues related to carrying out monitoring and evaluation, and the varying completion of practices recommended by established PSP literature. We propose four processes as integral to maximizing PSP’s usability for end users and recommend these areas for further study: identifying social imbalances throughout the PSP process, recognizing bias as inherent to PSP, explicitly addressing, and incorporating uncertainty, and allocating resources for monitoring and evaluation.

Erudite Survivor: Usability Testing of a Gamification-based Mobile App for Disaster Awareness Among Children

Numerous peer-reviewed studies describe designs for a mobile application for children based on gamification and catastrophe awareness. However, few articles give extensive evaluations of their solutions’ usability for end users. The purpose of this study is to present the usability assessment findings of a mobile app aimed at raising children’s catastrophe awareness in Malaysia. The study includes 12 target users to test the usability of the Erudite Survivor mobile app prototype. The test evaluation contains a pre-test, post-task and post-test questionnaire (Post-Study System Usability Questionnaire). The primary sections of this Post-Study System Usability Questionnaire are Overall Satisfaction, System Usefulness, Information Quality, and Interface Quality. The Post-Study System Usability Questionnaire revealed the following mean scores: Overall satisfaction (2.09), System Usefulness (2.03), Information Quality (2.06), and Interface Quality (2.25). Users of the prototype were satisfied with the application because the score was near to 1. This application’s capacity to teach children about disaster awareness might be viewed as the initial step toward a future improvement of gamification-based disaster education among Malaysian children.

A Smart Digital Health Platform to Enable Monitoring of Quality of Life and Frailty in Older Patients with Cancer: A Mixed-Methods, Feasibility Study Protocol

ObjectivesLifeChamps is an EU Horizon 2020 project that aims to create a digital platform to enable monitoring of health-related quality of life and frailty in patients with cancer over the age of 65. Our primary objective is to assess feasibility, usability, acceptability, fidelity, adherence, and safety parameters when implementing LifeChamps in routine cancer care. Secondary objectives involve evaluating preliminary signals of efficacy and cost-effectiveness indicators. Data SourcesThis will be a mixed-methods exploratory project, involving four study sites in Greece, Spain, Sweden, and the United Kingdom. The quantitative component of LifeChamps (single-group, pre-post feasibility study) will integrate digital technologies, home-based motion sensors, self-administered questionnaires, and the electronic health record to (1) enable multimodal, real-world data collection, (2) provide patients with a coaching mobile app interface, and (3) equip healthcare professionals with an interactive, patient-monitoring dashboard. The qualitative component will determine end-user usability and acceptability via end-of-study surveys and interviews. ConclusionThe first patient was enrolled in the study in January 2023. Recruitment will be ongoing until the project finishes before the end of 2023. Implications for Nursing PracticeLifeChamps provides a comprehensive digital health platform to enable continuous monitoring of frailty indicators and health-related quality of life determinants in geriatric cancer care. Real-world data collection will generate “big data” sets to enable development of predictive algorithms to enable patient risk classification, identification of patients in need for a comprehensive geriatric assessment, and subsequently personalized care.

TRAAC framework to improve regulatory acceptance and wider usability of tools and methods for safe innovation and sustainability of manufactured nanomaterials

There has been an increasing use of advanced materials, particularly manufactured nanomaterials, in industrial applications and consumer products in the last two decades. It has instigated concerns about the sustainability, in particular, risks and uncertainties regarding the interactions of the manufactured nanomaterials with humans and the environment. Consequently, significant resources in Europe and beyond have been invested into the development of tools and methods to support risk mitigation and risk management, and thus facilitate the research and innovation process of manufactured nanomaterials. The level of risk analysis is increasing, including assessment of socio-economic impacts, and sustainability aspects, moving from a conventional risk-based approach to a wider safety-and-sustainability-by-design perspective. Despite these efforts on tools and methods development, the level of awareness and use of most of such tools and methods by stakeholders is still limited. Issues of regulatory compliance and acceptance, reliability and trust, user-friendliness and compatibility with the users' needs are some of the factors which have been traditionally known to hinder their widespread use. Therefore, a framework is presented to quantify the readiness of different tools and methods towards their wider regulatory acceptance and downstream use by different stakeholders. The framework diagnoses barriers which hinder regulatory acceptance and wider usability of a tool/method based on their Transparency, Reliability, Accessibility, Applicability and Completeness (TRAAC framework). Each TRAAC pillar consists of criteria which help in evaluating the overall quality of the tools and methods for their (i) compatibility with regulatory frameworks and (ii) usefulness and usability for end-users, through a calculated TRAAC score based on the assessment. Fourteen tools and methods were assessed using the TRAAC framework as proof-of-concept and for user variability testing. The results provide insights into any gaps, opportunities, and challenges in the context of each of the 5 pillars of the TRAAC framework. The framework could be, in principle, adapted and extended to the evaluation of other type of tools & methods, even beyond the case of nanomaterials.

Integrating machine learning predictions for perioperative risk management: Towards an empirical design of a flexible-standardized risk assessment tool

BackgroundSurgical patients are complex, vulnerable, and prone to postoperative complications that can potentially be mitigated with quality perioperative risk assessment and management. Several institutions have incorporated machine learning (ML) into their patient care to improve awareness and support clinician decision-making along the perioperative spectrum. Recent research suggests that ML risk prediction can support perioperative patient risk monitoring and management across several situations, including the operating room (OR) to intensive care unit (ICU) handoffs. ObjectivesOur study objectives were threefold: (1) evaluate whether ML-generated postoperative predictions are concordant with clinician-generated risk rankings for acute kidney injury, delirium, pneumonia, deep vein thrombosis, and pulmonary embolism, and establish their associated risk factors; (2) ascertain clinician end-user suggestions to improve adoption of ML-generated risks and their integration into the perioperative workflow; and (3) develop a user-friendly visualization format for a tool to display ML-generated risks and risk factors to support postoperative care planning, for example, within the context of OR-ICU handoffs. MethodsGraphical user interfaces for postoperative risk prediction models were assessed for end-user usability through cognitive walkthroughs and interviews with anesthesiologists, surgeons, certified registered nurse anesthetists, registered nurses, and critical care physicians. Thematic analysis relying on an explanation design framework was used to identify feedback and suggestions for improvement. Results17 clinicians participated in the evaluation. ML estimates of complication risks aligned with clinicians' independent rankings, and related displays were perceived as valuable for decision-making and care planning for postoperative care. During OR-ICU handoffs, the tool could speed up report preparation and remind clinicians to address patient-specific complications, thus providing more tailored care information. Suggestions for improvement centered on electronic tool delivery; methods to build trust in ML models; modifiable risks and risk mitigation strategies; and additional patient information based on individual preferences (e.g., surgical procedure). ConclusionsML estimates of postoperative complication risks can provide anticipatory guidance, potentially increasing the efficiency of care planning. We have offered an ML visualization framework for designing future ML-augmented tools and anticipate the development of tools that recommend specific actions to the user based on ML model output.Statement of Significance.•Problem: Implementation of machine-learning (ML) output within clinical workflows is not trivial.•What is already known: Barriers to ML implementation and appropriate use within clinical workflows include clinicians’ limited understanding of ML, wariness of ML output, and lack of ML output actionability.•What this paper adds: Qualitative end-user evaluation of postoperative risk predictions found high-levels of agreement between rankings of clinicians' manual risk rankings and ML-augmented risk estimates. The evaluation also highlighted design (functional and visual) enhancements for translational implementation of ML models within clinical workflows.

Open Source District Heating Modeling Tools—A Comparative Study

Heating networks are highly relevant for the achievement of climate protection goals of urban energy systems. This is due to their high renewable energy potential combined with high plant efficiency and utilization rates. For the optimal integration and sector coupling of heating networks in holistic urban energy systems, open source energy system modeling tools are highly recommended. In this contribution, two open source approaches (the “Spreadsheet Energy System Model Generator”-integrated DHNx-Python module (DHNx/SESMG) and Thermos) are theoretically compared, and practically applied to a real-world energy system. Deviations within the results can be explained by incorrectly pre-defined parameters within Thermos and cannot be adjusted by the modeler. The simultaneity is underestimated in the case study by Thermos by more than 20%. This results in undersized heating plant capacities and a 50% higher number of buildings connected to the network. However, Thermos offers a higher end-user usability and over 100 times faster solving. DHNx/SESMG, in contrast, offers the possibility to adjust more model parameters individually and consider multiple energy sectors. This enables a holistic modeling of urban energy systems and the model-based optimization of multi-sectoral synergies.

Quantification of Digital Body Maps for Pain: Development and Application of an Algorithm for Generating Pain Frequency Maps.

BackgroundPain is an unpleasant sensation that signals potential or actual bodily injury. The locations of bodily pain can be communicated and recorded by freehand drawing on 2D or 3D (manikin) surface maps. Freehand pain drawings are often part of validated pain questionnaires (eg, the Brief Pain Inventory) and use 2D templates with undemarcated body outlines. The simultaneous analysis of drawings allows the generation of pain frequency maps that are clinically useful for identifying areas of common pain in a disease. The grid-based approach (dividing a template into cells) allows easy generation of pain frequency maps, but the grid’s granularity influences data capture accuracy and end-user usability. The grid-free templates circumvent the problem related to grid creation and selection and provide an unbiased basis for drawings that most resemble paper drawings. However, the precise capture of drawn areas poses considerable challenges in producing pain frequency maps. While web-based applications and mobile-based apps for freehand digital drawings are widely available, tools for generating pain frequency maps from grid-free drawings are lacking.ObjectiveWe sought to provide an algorithm that can process any number of freehand drawings on any grid-free 2D body template to generate a pain frequency map. We envisage the use of the algorithm in clinical or research settings to facilitate fine-grain comparisons of human pain anatomy between disease diagnosis or disorders or as an outcome metric to guide monitoring or discovery of treatments.MethodsWe designed a web-based tool to capture freehand pain drawings using a grid-free 2D body template. Each drawing consisted of overlapping rectangles (Scalable Vector Graphics <rect> elements) created by scribbling in the same area of the body template. An algorithm was developed and implemented in Python to compute the overlap of rectangles and generate a pain frequency map. The utility of the algorithm was demonstrated on drawings obtained from 2 clinical data sets, one of which was a clinical drug trial (ISRCTN68734605). We also used simulated data sets of overlapping rectangles to evaluate the performance of the algorithm.ResultsThe algorithm produced nonoverlapping rectangles representing unique locations on the body template. Each rectangle carries an overlap frequency that denotes the number of participants with pain at that location. When transformed into an HTML file, the output is feasibly rendered as a pain frequency map on web browsers. The layout (vertical-horizontal) of the output rectangles can be specified based on the dimensions of the body regions. The output can also be exported to a CSV file for further analysis.ConclusionsAlthough further validation in much larger clinical data sets is required, the algorithm in its current form allows for the generation of pain frequency maps from any number of freehand drawings on any 2D body template.

The Android Platform Security Model

Android is the most widely deployed end-user focused operating system. With its growing set of use cases encompassing communication, navigation, media consumption, entertainment, finance, health, and access to sensors, actuators, cameras, or microphones, its underlying security model needs to address a host of practical threats in a wide variety of scenarios while being useful to non-security experts. The model needs to strike a difficult balance between security, privacy, and usability for end users, assurances for app developers, and system performance under tight hardware constraints. While many of the underlying design principles have implicitly informed the overall system architecture, access control mechanisms, and mitigation techniques, the Android security model has previously not been formally published. This article aims to both document the abstract model and discuss its implications. Based on a definition of the threat model and Android ecosystem context in which it operates, we analyze how the different security measures in past and current Android implementations work together to mitigate these threats. There are some special cases in applying the security model, and we discuss such deliberate deviations from the abstract model.

A Hybrid Model of Hesitant Fuzzy Decision-Making Analysis for Estimating Usable-Security of Software

The main idea for practicing research towards security is to maintain usability of the software as well. This can be achieved by making less complex and high secure software. Though considerable efforts have been made in this context by the developers and security experts, the standard of usable-security is not the same as it should be. Therefore, it is important to research more on the most conversant mechanisms for assessing and increasing the usable-security of software. The quantitative evaluation of usability and security is an important part in increasing the level of secure software. This paper, in particular, is intended to estimate the usable-security of software and achieve the objective of developing software with optimum security while retaining its usability. The decision-makers often find it difficult to integrate security and usability. Hence this paper approaches the integration of usability and security with its contributing attributes. Also, in this league, the present research study suggests a novel technique which is the hybrid of Analytic Hierarchy Process (AHP), Hesitant Fuzzy (HF) sets, and Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) to significantly assess the usability along with security. To test the usefulness of the proposed approach hybrid HF-AHP-TOPSIS technique, the researchers applied it onto software developed for Babasaheb Bhimrao Ambedkar University in India. Furthermore, the procedural sensitivity was also achieved by using the various versions of the method. The findings of the usability along with security assessment insist that this inventive hybrid procedure would be the most conversant mechanism for determining the usable-security of software. Further these findings will be helpful in managing security without affecting the usability for end user.

A Mobile Health Intervention for HIV Prevention Among Racially and Ethnically Diverse Young Men: Usability Evaluation.

BackgroundMobile health (mHealth) apps have the potential to be a useful mode of delivering HIV prevention information, particularly for young men (13-24 years) who account for 21% of new HIV diagnoses in the United States. We translated an existing evidence-based, face-to-face HIV prevention curriculum into a portable platform and developed a mobile Web app: MyPEEPS Mobile.ObjectiveThe purpose of this study was to assess the usability of MyPEEPS Mobile from both expert and end user perspectives.MethodsWe conducted a heuristic evaluation with five experts in informatics to identify violations of usability principles and end user usability testing with 20 young men aged 15 to 18 years in New York, NY, Birmingham, AL, and Chicago, IL to identify potential obstacles to their use of the app.ResultsMean scores of the overall severity of the identified heuristic violations rated by experts ranged from 0.4 and 2.6 (0=no usability problem to 4=usability catastrophe). Overall, our end users successfully completed the tasks associated with use case scenarios and provided comments/recommendations on improving usability of MyPEEPS Mobile. The mean of the overall Post-Study System Usability Questionnaire scores rated by the end users was 1.63 (SD 0.65), reflecting strong user acceptance of the app.ConclusionsThe comments made by experts and end users will be used to refine MyPEEPS Mobile prior to a pilot study assessing the acceptability of the app across diverse sexual minority young men in their everyday lives.

A multi-level usability evaluation of mobile health applications: A case study.

To report a methodological approach for the development of a usable mHealth application (app). This work was guided by a 3-level stratified view of health information technology (IT) usability evaluation framework. We first describe a number of methodologies for operationalizing each level of the framework. Following the description of each methodology, we present a case study which illustrates the use of our preferred methodologies for the development of a mHealth app. At level 1 (user-task), we applied a card sorting technique to guide the information architecture of a mobile HIV symptom self-management app, entitled mVIP. At level 2 (user-task-system), we conducted a usability evaluation of mVIP in a laboratory setting through end-user usability testing and heuristic evaluation with informatics experts. At level 3 (user-task-system-environment), usability of mVIP was evaluated in a real-world setting following the use of the app during a 3-month trial. The 3-level usability evaluation guided our work exploring in-depth interactions between the user, task, system, and environment. Integral to the findings from the 3-level usability evaluation, we iteratively refined the app's content, functionality, and interface to meet the needs of our intended end-users. The stratified view of the health IT usability evaluation framework is a useful methodological approach for the design, development, and evaluation of mHealth apps. The methodological recommendations for using the theoretical framework can inform future usability studies of mHealth apps.

Comparison of Cognitive Modeling and User Performance Analysis for Touch Screen Mobile Interface Design

ABSTRACTThe aim of this study is to analyze and comparatively evaluate the usability of touch screen mobile applications through cognitive modeling and end-user usability testing methodologies. The study investigates the accuracy of the estimated results of a cognitive model produced for touch screen mobile phone interfaces. A mobile wallet application was chosen as the mobile software. The CogTool modeling tool was used as the cognitive modeling method. Eight tasks were determined and user tests were conducted in a usability laboratory with 10 participants. The tasks were compared on the basis of step time and total task completion time. This study reveals that CogTool gives approximate estimations with actual user performance on touch screen mobile phone application interfaces. However, if there are special cases in the tasks such that users are very accustomed to the steps or decision-making that is involved in the tasks, the “Think Operation” in CogTool should be modified.

Learning in mobility with Context4Learning: developing a context-aware mobile learning application

The adaptation possibilities provided by mobile devices strive to encompass and motivate students' education process by providing learning resources and activities adapted to their specific needs, characteristics, conditions and circumstances. Context4Learning is a mobile application developed by the team behind this paper. Developed for Android, the app detects user location, amount of time available and physical activity being performed to then send users the corresponding notifications regarding learning resources and activities with which he/she may advance in a given educational program. Currently, a pilot project has been launched. The assessment and contributions by the pilot user group encourage us to continue our work. Their experience has proven that work is still needed to achieve perfection in the functioning of the mechanism and greater app usability for end users.

A Conceptual Framework for Creating Cartographic Representations in SDI Environments

The low awareness of the communication function of maps within the Geospatial Data Infrastructure (SDI) community has prompted PhD research (Hopfstock, 2010) to explore how the concept of user-oriented map design can be utilized to improve the SDI usability for end users. In particular, the research aimed at providing a methodology for creating effective cartographic representations and design solutions for SDI relevant user types that facilitate gaining useful Geographic Information (GI) at the interface between SDI and their users as a result of the map viewing and interpretation process. This paper outlines the empirical research and highlights the main findings.

Context-sensitive user interfaces for semantic services

Service-centric solutions usually require rich context to fully deliver and better reflect on the underlying applications. We present a novel use of context in the form of customized user interface services with the concept of User Interface as a Service (UIaaS). UIaaS takes user profiles as input to generate context-aware interface services. Such interface services can be used as context to augment semantic services with contextual information leading to UIaaS as a Context (UIaaSaaC). The added serendipitous benefit of the proposed concept is that the composition of a customized user interface with the requested service is performed by the service composition engine, as is the case with any other services. We use a special-purpose language (called User Interface Description Language (UIDL)) to model and realize user interfaces as services. We use a real-life e-government application, human services delivery for the citizens, as a proof-of-concept. We also present a comprehensive evaluation of the proposed approach using a functional evaluation and a nonfunctional evaluation consisting of an end user usability test and expert usability reviews.

The MERG Suite: Tools for discovering competencies and associated learning resources

BackgroundAs the demands for competency-based education grow, the need for standards-based tools to allow for publishing and discovery of competency-based learning content is more pressing. This project focused on developing federated discovery services for competency-based medical e-learning content.MethodsWe built a tool suite for authoring and discovery of medical e-learning metadata. The end-user usability of the tool suite was evaluated through a web-based survey.ResultsThe suite, implemented as an open-source system, was evaluated to identify areas for improvement.ConclusionThe MERG suite is a starting point for organizations implementing competency-based e-learning resources.

Feature selection for content-based image retrieval

In this article, we propose a novel system for feature selection, which is one of the key problems in content-based image indexing and retrieval as well as various other research fields such as pattern classification and genomic data analysis. The proposed system aims at enhancing semantic image retrieval results, decreasing retrieval process complexity, and improving the overall system usability for end-users of multimedia search engines. Three feature selection criteria and a decision method construct the feature selection system. Two novel feature selection criteria based on inner-cluster and intercluster relations are proposed in the article. A majority voting-based method is adapted for efficient selection of features and feature combinations. The performance of the proposed criteria is assessed over a large image database and a number of features, and is compared against competing techniques from the literature. Experiments show that the proposed feature selection system improves semantic performance results in image retrieval systems.

Rules and ontologies in support of real-time ubiquitous application

The focus of this paper is the practical evaluation of the challenges and capabilities of combination of ontologies and rules in the context of realtime ubiquitous application. The ec(h)o project designed a platform to create a museum experience that consists of a physical installation and an interactive virtual layer of three-dimensional soundscapes that are physically mapped to the museum displays. The retrieval mechanism is built on the user model and conceptual descriptions of sound objects and museum artifacts. The rule-based user model was specifically designed to work in environments where the rich semantic descriptions are available. The retrieval criteria are represented as inference rules that combine knowledge from psychoacoustics and cognitive domains with compositional aspects of interaction. Evaluation results both from the laboratory and museum deployment testing are presented together with the end user usability evaluations. We also summarize our findings in the lessons learned that provide a transferable generic knowledge for similar type of applications. The ec(h)o proved that ontologies and rules provide an excellent platform for building a highly-responsive context-aware interactive application.

Rules and Ontologies in Support of Real-Time Ubiquitous Application

The focus of this paper is the practical evaluation of the challenges and capabilities of combination of ontologies and rules in the context of realtime ubiquitous application. The ec(h)o project designed a platform to create a museum experience that consists of a physical installation and an interactive virtual layer of three-dimensional soundscapes that are physically mapped to the museum displays. The retrieval mechanism is built on the user model and conceptual descriptions of sound objects and museum artifacts. The rule-based user model was specifically designed to work in environments where the rich semantic descriptions are available. The retrieval criteria are represented as inference rules that combine knowledge from psychoacoustics and cognitive domains with compositional aspects of interaction. Evaluation results both from the laboratory and museum deployment testing are presented together with the end user usability evaluations. We also summarize our findings in the lessons learned that provide a transferable generic knowledge for similar type of applications. The ec(h)o proved that ontologies and rules provide an excellent platform for building a highly-responsive context-aware interactive application.