Human-system Interaction Research Articles

Review of the opportunities and challenges to accelerate mass‐scale application of smart grids with large‐language models

AbstractSmart grids represent a paradigm shift in the electricity industry, moving from traditional one‐way systems to more dynamic, interconnected networks. These grids are characterised by their intelligent automation, robust structure, and enhanced interaction with customers, backed by comprehensive monitoring and data analytics. The key of this transformation is the integration of data‐driven methods into smart grids. Compared to previous big data solutions, large language models (LLMs), with their advanced generalisation abilities and multi‐modal competencies, are crucial in effectively managing and integrating diverse data sources. They address challenges such as data inconsistency, inadequate quality, and heterogeneity, thereby enhancing the operational efficiency and reliability of smart grids. Furthermore, at the system level, LLMs improve human–system interactions, making smart grids more user‐friendly and intuitive. Last but not the least, the structure of LLMs performs inherent advantages in bolstering system security and privacy, alongside in resolving issues related to system compatibility and integration. The paper reviews the data‐empowered smart grids and for the first time finds and proposes opportunities and future directions for adopting LLMs to accelerate the mass‐scale application of Smart Grids.

Exploring the Impact of AI Enhancement on the Sports App Community: Analyzing Human-Computer Interaction and Social Factors Using a Hybrid SEM-ANN Approach

The iteration of AI technology has enhanced the industry competitiveness of sports apps and brought a new emotional experience to users. However, the community function of AI-enabled sports apps has not yet received sufficient attention. This study adopts the technology acceptance model framework through the human-computer interaction and social psychology perspectives, incorporating perceived interactivity (human-human interaction, human-information interaction, human-system interaction), user experience (functional experience, content experience, and emotional experience), attachment theory (emotional attachment, emotional loyalty), social identity, social trust, and satisfaction variables to construct a theoretical model. It aims to solve the problem of how AI can enhance users’ emotional interaction, community stickiness, and ultimately social interaction satisfaction in sports apps. Through literature review, theoretical analysis, and case study, the Chinese “KEEP” APP was identified as the research object, and the questionnaire was constructed with an improved test scale. The study used a cross-sectional sampling method to collect data from Chinese online platforms for three months (N = 520) and tested the hypotheses in the theoretical model through structural equation modeling and artificial neural networks. The results of the study show that (1) emotional attachment, emotional loyalty, social identity, and social trust positively and significantly affect user satisfaction; (2) emotional attachment and social identity positively and significantly affect human-human interaction; (3) perceived usefulness, social identity, and social trust significantly construct an influence loop on the internal path. Compared with existing studies, this study proposes a multidimensional innovative framework that reveals the deeper influence mechanism of AI technology in sports APP communities and makes up for the shortcomings of existing studies in terms of users’ emotional interactions and community stickiness. Through this study, sports APP developers and operators, marketers, AI technology developers, as well as sports enthusiasts, and general users can obtain specific practical suggestions to further optimize the design and functionality of sports apps, and enhance the overall user experience and satisfaction.

Flexible wide-range, sensitive three-axis pressure sensor array for robotic grasping feedback

Flexible pressure sensors capable of detecting normal and tangential forces through physical contact have garnered considerable interest in the realm of human-interactive systems. However, simultaneous detection of multi-directional forces is still a challenge for current research. Herein, a capacitive flexible pressure sensor based on a sandwich structure for three-dimensional force detection is proposed. The fabrication process of the sensor array is straightforward, capable of effectively distinguishing between normal and tangential forces. Polyimide (PI) serves as the flexible substrate for depositing the metal electrode pattern, while Polydimethylsiloxane (PDMS) acts as the intermediate dielectric layer material and the three-dimensional force conduction block. Through a comparative study of the thickness of the hollow dielectric layer, a pressure sensor with superior performance was prepared, featuring high sensitivity across a wide working range. Test results demonstrate its capability to detect normal forces ranging from 0 to 46 N (0–520 kPa) with a sensitivity of 0.442 N−1 (0.031 kPa−1) and tangential forces from 0 to 10 N with a sensitivity of 0.08 N−1 (X-axis) and 0.07 N−1 (Y-axis). The designed acquisition system can simultaneously gather data from 6 sensor arrays, totaling 240 channels, with a response time of 11 ms. This sensor array, characterized by flexibility, versatility, and a wide range, is suitable for applications in robot tactile perception.

Digital Human Intelligent Interaction System Based on Multimodal Pre-training Mode

ABSTRACT As the forefront of human–computer intelligent interaction, digital humans have increasingly diverse application scenarios, but also face many challenges. In order to enhance the intelligence level and interaction capability of digital human, the study first optimizes the traditional UniLM model, then introduces the mechanism of multi-head attention, and mitigates the exposure bias by using adversarial training and random replacement of decoder to design an improved UniLM model, and finally applies the improved model to the digital human management system. The results show that the precision rate of the improved UniLM model is improved by 7.68%, 6.4%, and 4.96%, the recall rate is improved by 11.94%, 9.69%, and 8.83%, and the F1-score is improved by 8.34%, 6.41%, and 7.68% compared with the other three models, which proves that it has a better precision rate, robustness, and generalization ability. The perplexity of the improved UniLM model is 135, 95, 76, 71, and 55 under five text lengths, which is significantly lower than the other models, proving that its text generation ability is better. The above results demonstrate the performance of the research-designed digital human system based on the Improved UniLM model, which provides a direction for the further development of digital human technology.

Heuristics and biases in human–algorithm interaction and hotel revenue management override decision-making

Purpose This study aims to provide insights into human–algorithm interaction in revenue management (RM) decision-making and to uncover the underlying heuristics and biases of overriding systems’ recommendations. Design/methodology/approach Following constructivist traditions, 20 in-depth interviews were conducted with revenue optimisers, analysts, managers and directors with vast experience in over 25 markets and working with different RM systems (RMSs) at the property and corporate levels. The hermeneutics approach was used to interpret and make meaning of the participants’ lived experiences and interactions with RMSs. Findings The findings explain the nature of the interaction between RM professionals and RMSs, the cognitive mechanism by which the system users judgementally adjust or override its recommendations and the heuristics and biases behind override decisions. Additionally, the findings reveal the individual decision-maker characteristics and organisational factors influencing human–algorithm interactions. Research limitations/implications Although the study focused on human–system interaction in hotel RM, it has larger implications for integrating human judgement into computerised systems for optimal decision-making. Practical implications The study findings expose human biases in working with RMSs and highlight the influencing factors that can be addressed to achieve effective human–algorithm interactions. Originality/value The study offers a holistic framework underpinned by the organisational role and expectation confirmation theories to explain the cognitive mechanisms of human–system interaction in managerial decision-making.

Smart All-Hazard Responses Framework (SARF): Human Factors and Ergonomics Approach

The integration of Human Factors and Ergonomics (HFE) into emergency responses is crucial due to frequent human errors and inadequate human-system interactions which impede effective emergency and disaster management. Therefore, this research aims to develop and validate the Smart All-hazards Response Framework (SARF) that systematically incorporates HFE principles across all aspects of disaster management, from planning through post-response analysis. The framework focuses on reducing human errors, enhancing human-system interactions, and boosting system resilience. The research methodology includes a detailed literature review, the development of the HFE-integrated framework, and rigorous testing through experimental studies and simulations across various scenarios. Key questions address the integration of HFE to optimize system performance and the impact of ergonomic interventions on responder safety in high-risk environments. By advancing a holistic, human-centered approach to emergency and disaster management, this study aims to significantly enhance the effectiveness, safety, and adaptability of responses to all kinds of hazards.

1B4-2 Issues on human-system interaction with visual displays and Image Safety, and their International Standardizations

1B4-2 Issues on human-system interaction with visual displays and Image Safety, and their International Standardizations

Exploring Trade-offs in Explainable AI

Machine Learning (ML) models are increasingly used in systems that involve physical human interaction or decision-making systems that impact human health and safety. Ensuring that these systems are safe and reliable is an important topic of current AI research. For many ML models it is unclear how a prediction (output) is arrived at from the provided features (input). Critical systems cannot blindly trust the predictions of such "black box" models, but instead need additional reassurance via insight into the model's reasoning. A range of methods exist within the field of Explainable AI (XAI) to make the reasoning of black box ML models more understandable and transparent. The explanations provided by XAI methods may be evaluated in a number of (competing) ways. In this paper, we investigate the trade-off between selected metrics for an XAI method called UnRAvEL, which is similar to the popular LIME approach. Our results show that by weighting the terms within the acquisition function used in UnRAvEL, different trade-offs can be achieved.

Examining the Use of Intelligent Conversational Voice-Assistants for Improved Mobility Behavior of Older Adults in Smart Cities

Intelligent Conversational Voice-assistants (ICV) can interpret the speech of human and respond back using synthesized voices and are increasingly employed for interacting with different applications. ICV offers promising tool to address the increasing mobility accessibility challenge faced by older people. Yet, irrespective of the increased ageing population, studies that employed ICV to improve the mobility, walkability, and wayfinding of older people are scarce. This is because the development of ICV in the mobility domain presents several designs, technical, and linguistic challenges. A systematic literature review was adopted grounded on secondary data from the literature and descriptive analysis was employed. Grounded on user-centred design perspective based on the “ability-based design framework” and “ISO 9241-110 framework for ergonomics of human-system interaction.” Therefore, this study describes the requirement specifications needed to design an ICV suitable for supporting the mobility behavior of older people. Findings from this paper identifies the perceived factors that influences the use of ICV by older people. Additionally, findings from this study discusses how ICV that simulates human-like behavior can provide personalized mobility, walkability, and wayfinding guide when older people walk in cities. This study provides recommendations for the implementation of a user-centred ICV for safe, independent, accessible mobility for older people in urban environment.

Design and Optimization of Human-Computer Interaction System for Education Management Based on Artificial Intelligence

The continuous improvement and refinement of artificial intelligence (AI) technology has facilitated the broader application of human-computer interaction in the field of education management. The construction of an educational management human-computer interaction system based on AI technology can optimize and improve key parameters of educational management human-computer interaction scenarios, thereby creating a more comprehensive mobile learning (m-learning) application system. This paper is based on AI technology, analyzing gesture semantics and speech semantics, and combining fusion algorithms to construct an education management human interaction system. The performance changes of the system were compared with real experimental operations and the NOBOOK platform analysis. The results show that the education management human-computer interaction system constructed in this article can enhance the m-learning experience of participants. It ensures high recognition accuracy, leading to higher scores in all dimensions of indicator evaluation. Therefore, as one of the crucial forms of m-learning, the human-computer interaction system for education management based on AI can establish a foundation for the further enhancement and development of education management.

Portable Head-Mounted System for Mobile Forearm Tracking.

Computer vision (CV)-based systems using cameras and recognition algorithms offer touchless, cost-effective, precise, and versatile hand tracking. These systems allow unrestricted, fluid, and natural movements without the constraints of wearable devices, gaining popularity in human-system interaction, virtual reality, and medical procedures. However, traditional CV-based systems, relying on stationary cameras, are not compatible with mobile applications and demand substantial computing power. To address these limitations, we propose a portable hand-tracking system utilizing the Leap Motion Controller 2 (LMC) mounted on the head and controlled by a single-board computer (SBC) powered by a compact power bank. The proposed system enhances portability, enabling users to interact freely with their surroundings. We present the system's design and conduct experimental tests to evaluate its robustness under variable lighting conditions, power consumption, CPU usage, temperature, and frame rate. This portable hand-tracking solution, which has minimal weight and runs independently of external power, proves suitable for mobile applications in daily life.

Analysis of human errors in human-autonomy collaboration in autonomous ships operations through shore control experimental data

Human-autonomy collaboration plays a pivotal role in the development of Maritime autonomous surface ships (MASS), as Shore control center (SCC) operators may engage in the control loop by directly operating the MASS, or, in the supervisory loop, monitoring the MASS and taking over control when needed. Thus, effective human performance during takeover control and operation is crucial for the safety of MASS operations. However, since the MASS is still in the early phase of development, the mechanism of human errors is unknown, and the data on human-autonomy collaborative operation is scarce. Human reliability analysis (HRA) aims to assess human errors qualitatively and quantitatively, and is widely used in various complex systems to help safety analysis. This study is dedicated to incorporating advanced HRA methods elements to identify and quantify human errors during taking over control and operation of a MASS in collision avoidance scenarios. It presents virtual experimental results, combined with theoretical human error identification and assessment methods. At first, we apply the Human-System Interaction in Autonomy (H-SIA) method to identify potential human errors; secondly, we identify relevant Performance Shaping Factors (PSFs) including Experience, Boredom, Task complexity, Available time and Pre-warning, and performance measures of the human errors, and implement them in the virtual experiment based on a full-scale autonomous ferry research vessel called milliAmpere2. Finally, we build a Bayesian Network (BN) to present causal and probabilistic relationships between PSFs and human errors through experimental data. The results show that available time has the highest impact on takeover performance of operators, followed by task complexity and pre-warning. Boredom does not present a significant sole impact unless combined with available time. Experience does not show a significant impact on human performance. In addition to the relevance of the human errors analysis to the safe development and operational design of MASS, the developed method benefits other human-autonomy collaborative systems. The developed BN model shows adaptability to assess human error probabilities, and the practical significance of integrating experimental data into the existing HRA methodologies for complex systems.

Interactive Learning System for Learning Calculus.

IT tools has brought a new perspective to collaborative learning where students do not just sit in a chair and swallow lecture content but instead participate in creating and sharing knowledge. However, calculus learning augmented reality application has limitation in promoting a human collaboration in learning. This research develops an interactive application for learning calculus that promotes human-system interaction via augmented reality (AR) and human-human interaction through chat functions. The study examines the effect of both interactivities on learning experience and how that learning experience affects the performance of learning. The research adopted a quasi-experimental study design and pre-post test data analysis to evaluate the effect of interactivities on learning experience and consequently the effect of learning experience on learning performance. The subjects were exposed to the developed application for learning the calculus chapter "Solid of Revolution" in a controlled environment. The study validated its research framework through partial least squares path modelling and tested three hypotheses via pre-and post-test evaluation. The results found that both interactivities affect learning experience positively; human-human interactivity has a higher impact than the human-system interactivity. It was also found that learning performance as part of the learning experience increased from pre-test to post-test.

Safety protocols in electro-mechanical installations: Understanding the key methodologies for ensuring human and system safety

This study systematically examines the safety protocols in electro-mechanical systems, focusing on methodologies, trends, and innovations that ensure human and system safety. The primary objective is to bridge the gap between current practices and the evolving demands of these systems, considering technological advancements and regulatory compliance. A systematic literature review methodology was employed, utilizing databases like IEEE Xplore, ScienceDirect, and Google Scholar. The search strategy involved keywords related to safety protocols and electro-mechanical systems, adhering to strict inclusion and exclusion criteria to ensure a focused analysis. Key findings reveal the growing importance of integrating advanced sensing technologies, artificial intelligence, and human factors into safety management. The study highlights the shift from reactive to proactive safety measures, emphasizing the need for continuous monitoring and predictive maintenance. Collaborative approaches involving various stakeholders, including industry and academia, are identified as crucial for enhancing safety. The future landscape of safety in electro-mechanical systems is expected to be significantly influenced by emerging technologies, leading to more sophisticated safety protocols. The study recommends continuous updating of safety protocols in line with technological advancements and changing regulatory landscapes. It also suggests interdisciplinary research and industry-academia collaboration to address complex safety challenges. Finally, the evolution of safety protocols in electro-mechanical systems is marked by technological advancements and a deeper understanding of human-system interaction. The study underscores the necessity for continuous improvement and adaptation of safety protocols, ensuring their effectiveness in safeguarding human lives and system sustainability.

Mental Model Matrix: Implications for System Design and Training

Mental models are characterized as a person’s mental representation of how something works, guiding how they process information, anticipate future events, and interact with devices and tools in complex sociotechnical systems. This paper introduces the Mental Model Matrix (MMM), a novel framework for conceptualizing mental models in the context of human-system interactions. The MMM framework partitions aspects of an individual’s knowledge about a target system into two primary dimensions (system and user), each containing beliefs regarding capabilities and limitations. The system-based dimension includes knowledge for the system’s parts, connections, and causal relationships (capabilities), as well as knowledge for how the system can fail (limitations). The user-based dimension incorporates tacit knowledge for working with the system (capabilities) along with an appreciation for one’s own confusions and misunderstandings (limitations). The MMM provides a conceptual framework and guidance for end-users, trainers, and system designers seeking to elicit and codify different dimensions of an individual’s mental model of the systems they work with. The MMM can be used to reveal knowledge gaps and misalignments between different stakeholders in an organization, which can facilitate the development of human-centered technologies and systems, and the creation of effective training programs.

Mechanochromic and ionic conductive cholesteric liquid crystal elastomers for biomechanical monitoring and human-machine interaction.

Cholesteric liquid crystal elastomers (CLCEs) that combine rubbery elasticity with structural colour from self-assembled helical nanostructures are of paramount importance for diverse applications such as biomimetic skins, adaptive optics and soft robotics. Despite great advances, it is challenging to integrate electrical sensing and colour-changing characteristics in a single CLCE system. Here, we report the design and synthesis of an ionic conductive cholesteric liquid crystal elastomer (iCLCE) through in situ Michael addition and free-radical photopolymerization of CLCE precursors on silane-functionalized polymer ionic liquid networks, in which robust covalent chemical bonding was formed at the interface. Thanks to superior mechanochromism and ionic conductivity, the resulting iCLCEs exhibit dynamic colour-changing and electrical sensing functions in a wide range upon mechanical stretching, and can be used for biomechanical monitoring during joint bending. Importantly, a capacitive elastomeric sensor can be constructed through facilely stacking iCLCEs, where the optical and electrical dual-signal reporting performance allows intuitive visual localization of pressure intensity and distribution. Moreover, proof-of-concept application of the iCLCEs has been demonstrated with human-interactive systems. The research disclosed herein can provide new insights into the development of bioinspired somatosensory materials for emerging applications in diverse fields such as human-machine interaction, prostheses and intelligent robots.

Time series synchronization in cross-recurrence networks: uncovering a homomorphic law across diverse complex systems

Exploring the synchronicity between time series, especially the similar patterns during extreme events, has been a focal point of research in academia. This is due to the fact that such special dependence occurring between pairs of time series often plays a crucial role in triggering emergent behaviors in the underlying systems and is closely related to systemic risks. In this paper, we investigate the relationship between the synchronicity of time series and the corresponding topological properties of the cross-recurrence network (CRN). We discover a positive linear relationship between the probability of pairwise time series event synchronicity and the corresponding CRN’s clustering coefficient. We first provide theoretical proof, then demonstrate this relationship through simulation experiments by coupled map lattices. Finally, we empirically analyze three instances from financial systems, Earth’s ecological systems, and human interactive behavioral systems to validate that this regularity is a homomorphic law in different complex systems. The discovered regularity holds significant potential for applications in monitoring financial system risks, extreme weather events, and more.

Bridging artificial intelligence and cognitive ergonomics: a call for synergy

There is considerable potential between the field of artificial intelligence (AI) and cognitive ergonomics. The purpose of this thesis is to support increasing attention to the synergy between artificial intelligence and cognitive ergonomics, as well as the urgent need to explore it in research and practice.Artificial intelligence (AI) has become an integral part of the modern workplace, transforming industries, increasing productivity, and streamlining processes. However, as AI technologies continue to evolve, it is essential to consider the human side of this digital transformation. Ergonomics, the science of designing workspaces and tools that fit the people who use them, is a critical element in ensuring that the integration of artificial intelligence into the workplace increases employee well-being, efficiency, and overall success.
 With its rapid advancements, artificial intelligence is revolutionizing various fields and industries, from healthcare to self-driving vehicles and financial services. While the potential benefits are obvious, the ethical, cognitive, and ergonomic implications are just as important. Cognitive ergonomics is a sub-branch of ergonomics that deals with the design and evaluation of systems and technologies that support human cognitive processes such as perception, attention, memory, and decision-making. The field of cognitive ergonomics, which primarily focuses on Optimizing human-system interaction is the key to ensuring that artificial intelligence systems are designed and implemented in a way that aligns with human capacities, abilities, and cognitive limitations . Current AI implementations typically adopt a technology-driven focus, and employees are expected to adapt to the technology. In this technology-driven focus, the performance, performance, and accuracy of AI are optimized, but these aspects are considered separately. This point of view raises various critical considerations that are often neglected in the design and implementation of advanced technologies and sometimes have disastrous consequences. From the point of view of ergonomics, the design of artificial intelligence should be transferred from a technology-oriented focus to a systemic and human perspective. By applying a focus on personnel, artificial intelligence should be meaningfully and safely designed and integrated into work processes with the aim of optimizing overall system performance and people's well-being.

Improving safety management in railway stations through a simulation-based digital twin approach

In the dynamic milieu of modern, bustling railway stations, the emphasis on safety and efficient pedestrian traffic management has never been more crucial, elevating the significance of Digital Twins (DTs). DTs bring transformative changes in the way we approach the design, operation, control, and maintenance of complex systems, such as transportation hubs. However, documented real-world applications are lacking, leaving scientists and practitioners without valuable insights on DT implementation. Drawing from a case study centered around a prominent Italian railway station, this paper showcases the prototype of a simulation-based DT that empowers station managers with capabilities such as pedestrian flow prediction, early congestion warnings, evacuation response planning, layout optimization, and intelligent gate management. The simulation-based DT acts as a sophisticated mirror reflecting the dynamics of a station's crowd, drawing data from its physical counterpart with a certain frequency, simulating the evolution of the system over a given time horizon, returning warning messages to the decision maker if issues are identified, and evaluating ex-ante different corrective solutions. The latter becomes crucial for controlling the physical world, providing a closed-loop system. Our case-based framework validates the effectiveness of methods and technical solutions engineered ad-hoc to achieve an acceptable tradeoff among accuracy, performance, and scalability. This includes addressing challenges related to the synchronization of the virtual and physical world, data integration and interoperability with existing systems, and human-system interaction. Ultimately, this contribution serves as a valuable resource, bridging the gap between theoretical concepts and tangible applications in the realm of transportation hub management.

ICTs and Smart Agriculture in Bangladesh: Way of Sustainable Development

This paper analyzes how the farmers of Bangladesh use the modern technology of information and communication for gathering different agricultural data. Farmers of Bangladesh are being encouraged to use smart devices of information and technology such as smart phones, android phones, laptops, desktops, electronic television for getting updates on different agricultural products, harvesting, and competitive markets. Information and communication technology includes all the devices with inbuilt human to human and device to human interaction systems. Thus, information and technology have a direct and indirect impact on rising farmers’ productivity and growth for sustainable management. This study sheds light on the understanding of the effect of information and communication technology in agricultural growth of Bangladesh for sustainable development with the help of conclusive research design and secondary method of data collection has adopted in this research for understanding of the importance of using information and technology in smart agriculture.