Human Computer Research Articles

Trust in artificial intelligence: Producing ontological security through governmental visions

With developments in artificial intelligence (AI) widely framed as security concern in both military and civilian realms, governments have turned their attention to regulating and governing AI. In a study of United States (US), Chinese, and European Union (EU) AI documents, we go beyond instrumental understandings of AI as a technological capability, which serves states’ self-interests and the maintenance of their (supra)national security. Our specific interest lies in how AI policies tap into both problem-solving approaches and affective registers to achieve both physical and ontological securities. We find that in governmental visions, AI is perceived as a capability that enhances societal and geopolitical interests while its risks are framed as manageable. This echoes strands within human–computer interaction that draw on human-centered perceptions of technology and assumptions about human–AI relationships of trust. Despite different cultural and institutional settings, the visions of future AI development are shaped by this (shared) understanding of human–AI interaction, offering common ground in the navigation of innovation policies.

Research on the Knowledge Structure and Sustainable Development Pathways of Artificial Intelligence from the Perspective of Technological Science

Achieving significant breakthroughs in both the fundamental theories and technological applications of artificial intelligence is essential for fostering its long-term development. Under the guidance of Professor Qian Xuesen’s theory of technological science, exploring the internal mechanisms of knowledge evolution in artificial intelligence holds profound theoretical and practical significance for promoting sustainable technological advancement. This study draws on literature from the Web of Science (WOS) database and employs methods such as knowledge mapping, natural language processing, clustering analysis, and citation analysis to outline the knowledge structure of the field, clarify the trajectory of sustainable development, and trace the technological genealogy of VR/AR technologies.This study divides the knowledge structure within the field of technological science into “basic theoretical knowledge—applied basic knowledge—applied knowledge”, enriching Qian’s theory of technological science from within and providing strong intellectual support and technological pathways for sustainable technological development in practice. Artificial intelligence encompasses 10 distinct knowledge domains, among which machine learning and deep learning constitute the basic theoretical knowledge, data intelligence, computer vision, and swarm intelligence are the applied basic knowledge, and image processing and human-computer intelligence are the applied knowledge. The development of VR/AR technology has formed two main sustainable development paths: “machine learning—data intelligence—intelligent systems—human computer intelligence”, and “deep learning—computer vision—image processing”.

Human Computation, Equitable, and Innovative Future of Work AI Tools

As we enter an era where the synergy between AI technologies and human effort is paramount, the Future of Work is undergoing a radical transformation. Emerging AI tools will profoundly influence how we work, the tools we use, and the very nature of work itself. The ’Human Computation, Equitable, and Innovative Future of Work AI Tools’ workshop at HCOMP’24 aims to explore groundbreaking solutions for developing fair and inclusive AI tools that shape how we will work. This workshop will delve into the collaborative potential of human computation and artificial intelligence in crafting equitable Future of Work AI tools. Participants will critically examine the current challenges in designing fair and innovative AI systems for the evolving workplace, as well as strategies for effectively integrating human insights into these tools. The primary objective is to foster a rich discourse on scalable, sustainable solutions that promote equitable Future of Work tools for all, with a particular focus on empowering marginalized communities. By bringing together experts from diverse fields, we aim to catalyze ideas that bridge the gap between technological advancement and social equity.

Trustworthy human computation: a survey

Human computation is an approach to solving problems that prove difficult using AI only, and involves the cooperation of many humans. Because human computation requires close engagement with both “human populations as users” and “human populations as driving forces,” establishing mutual trust between AI and humans is an important issue to further the development of human computation. This survey lays the groundwork for the realization of trustworthy human computation. First, the trustworthiness of human computation as computing systems, that is, trust offered by humans to AI, is examined using the RAS (reliability, availability, and serviceability) analogy, which define measures of trustworthiness in conventional computer systems. Next, the social trustworthiness provided by human computation systems to users or participants is discussed from the perspective of AI ethics, including fairness, privacy, and transparency. Then, we consider human–AI collaboration based on two-way trust, in which humans and AI build mutual trust and accomplish difficult tasks through reciprocal collaboration. Finally, future challenges and research directions for realizing trustworthy human computation are discussed.

Microexpression Recognition Method Based on ADP-DSTN Feature Fusion and Convolutional Block Attention Module

Microexpressions are subtle facial movements that occur within an extremely brief time frame, often revealing suppressed emotions. These expressions hold significant importance across various fields, including security monitoring and human–computer interaction. However, the accuracy of microexpression recognition is severely constrained by the inherent characteristics of these expressions. To address the issue of low detection accuracy regarding the subtle features present in microexpressions’ facial action units, this paper proposes a microexpression action unit detection algorithm, Attention-embedded Dual Path and Shallow Three-stream Networks (ADP-DSTN), that incorporates an attention-embedded dual path and a shallow three-stream network. First, an attention mechanism was embedded after each Bottleneck layer in the foundational Dual Path Networks to extract static features representing subtle texture variations that have significant weights in the action units. Subsequently, a shallow three-stream 3D convolutional neural network was employed to extract optical flow features that were particularly sensitive to temporal and discriminative characteristics specific to microexpression action units. Finally, the acquired static facial feature vectors and optical flow feature vectors were concatenated to form a fused feature vector that encompassed more effective information for recognition. Each facial action unit was then trained individually to address the issue of weak correlations among the facial action units, thereby facilitating the classification of microexpression emotions. The experimental results demonstrated that the proposed method achieved great performance across several microexpression datasets. The unweighted average recall (UAR) values were 80.71%, 89.55%, 44.64%, 80.59%, and 88.32% for the SAMM, CASME II, CAS(ME)3, SMIC, and MEGC2019 datasets, respectively. The unweighted F1 scores (UF1) were 79.32%, 88.30%, 43.03%, 81.12%, and 88.95%, respectively. Furthermore, when compared to the benchmark model, our proposed model achieved better performance with lower computational complexity, characterized by a Floating Point Operations (FLOPs) value of 1087.350 M and a total of 6.356 × 106 model parameters.

Synthesizing a three Layer Architectural Framework for Adaptive Intelligent User Interfaces for the Disabled Community

Knowledge acquisition has been a bottleneck in the design for efficient User Interfaces for Disabled Community. The best suited interfaces have been oriented towards adaptability that is the User Interface needs to adapt and configure itself according to the needs and likes of the user. Such user interfaces can arouse interest in the mind of the disabled and motivate them to become an asset to the nation, and use their intellect to contribute to the society. This study pioneers an innovative approach, aiming to develop User Interfaces (UIs) for the Disabled Community. The research introduces a comprehensive three-layer architectural framework employing Learning, Trust, and Knowledge Models. Unlike conventional approaches, this Architectural Framework is designed to learn not only from the Disabled but also from their Assistants/Guides, employing principles of knowledge engineering. While the former offers static context, the latter contributes dynamic insights by observing behaviors, responses, and actions of the Disabled. This symbiotic relationship forms the foundational basis for developing intelligent adaptive UI’s that not only accommodate but actively engage and motivate the Disabled Community. The proposed research methodology involves a meticulous gathering of data through extensive user studies and interactions. By using the Human Computer Interface technology, the study aims to develop a robust framework capable of comprehensively understanding the preferences, needs, and nuances of disabled users. The expected outcomes include the development of a prototype adaptive UI, validated through rigorous testing and feedback from the disabled community. Additionally, the study anticipates contributing a comprehensive set of guidelines and best practices for designing intelligent adaptive UIs, ensuring comfort and ease for impaired users, boosting their confidence and encouraging greater involvement in socio-development activities.

Comparative Analysis of Pre-trained based CNN-RNN Deep Learning Models on Anomaly-5 Dataset for Action Recognition

Action recognition in videos is one of the essential, challenging and active area of research in the field of computer vision that adopted in various applications including automated surveillance systems, security systems and human computer interaction. In this paper, we present an in-depth comparative analysis of five CNN-RNN models based on pre-trained networks such as InceptionV3, VGG16, MobileNetV2, ResNet152V2 and InceptionResNetV2 with recurrent LSTM units for action recognition on Anomaly-5 dataset. The performance of these models is analyzed and compared in terms of accuracy, precision, recall & F1-scores and computational efficiency. The CNN-RNN architectures we considered for analysis in this paper, the ResNet152V2 based CNN-RNN model exhibits better performance and achieved highest accuracy, precision, recall and F1-score equal to 92.20% due to its ability to capture more complex spatial features. This comparative analysis may guide the researchers in selecting appropriate models for real-world applications for action recognition. In addition of this, a new dataset is developed called Anomaly-5 that can helps as a valuable resource for training and evaluating action recognition algorithms.

Dual Leap Motion Controller 2: A Robust Dataset for Multi-view Hand Pose Recognition

This paper presents Multi-view Leap2 Hand Pose Dataset (ML2HP Dataset), a new dataset for hand pose recognition, captured using a multi-view recording setup with two Leap Motion Controller 2 devices. This dataset encompasses a diverse range of hand poses, recorded from different angles to ensure comprehensive coverage. The dataset includes real images with the associated precise and automatic hand properties, such as landmark coordinates, velocities, orientations, and finger widths. This dataset has been meticulously designed and curated to maintain a balance in terms of subjects, hand poses, and the usage of right or left hand, ensuring fairness and parity. The content includes 714,000 instances from 21 subjects of 17 different hand poses (including real images and 247 associated hand properties). The multi-view setup is necessary to mitigate hand occlusion phenomena, ensuring continuous tracking and pose estimation required in real human-computer interaction applications. This dataset contributes to advancing the field of multimodal hand pose recognition by providing a valuable resource for developing advanced artificial intelligence human computer interfaces.

Pen-based vibrotactile feedback rendering of surface textures under unconstrained acquisition conditions

Haptic rendering of surface textures enhances user immersion of human–computer interaction. However, strict input conditions and measurement methods limit the diversity of rendering algorithms. In this regard, we propose a neural network-based approach for vibrotactile haptic rendering of surface textures under unconstrained acquisition conditions. The method first encodes the interactions based on human perception characteristics, and then utilizes an autoregressive-based model to learn a non-linear mapping between the encoded data and haptic features. The interactions consist of normal forces and sliding velocities, while the haptic features are time–frequency amplitude spectrograms by short-time Fourier transform of the accelerations corresponding to the interactions. Finally, a generative adversarial network is employed to convert the generated time–frequency amplitude spectrograms into the accelerations. The effectiveness of the proposed approach is confirmed through numerical calculations and subjective experiences. This approach enables the rendering of a wide range of vibrotactile data for surface textures under unconstrained acquisition conditions, achieving a high level of haptic realism.

Invitations to play: designing for family engagement through STEAM exhibits in community settings

Intergenerational play between children and their family members provides a rich context that can inform family-centered interaction design. Despite research on the roles and perspectives in adult-child interactions, less is known about the design elements and features of voluntary interaction settings that provide opportunities for family members to collaborate and play in joint, balanced ways. To this end, we report the qualitative results of our design and implementation of three interactive STEAM exhibits installed in an early childhood education center over a six-month period. Our findings surface insights and tensions related to the design features’ impact on supporting family engagement in STEAM exhibits, emergent aspects of family-centered play, and supports needed to implement family-centered STEAM exhibits. Other designers, such as those who design with new technologies or explore human–computer interactions, can learn from our process of iteration and adjustment based on family interactions.

Working Memory Capacity for Mid-Air Gestures in Human–Computer Interaction

Mid-air gesture interactions have emerged as one of the predominant modalities used in human-computer interaction (HCI). However, extant research on gestures predominantly concentrates on the design of gestures for singular functionalities, which requires a comprehensive investigation into gesture interaction design from an integrative HCI system perspective. Working memory capacity plays a pivotal role in constraining the number of gestures accommodated within an HCI system. This study adopts the change detection paradigm to conduct an empirical investigation into the working memory capacities associated with mid-air gestures, aiming to elucidate the working memory capacity for such gestures. The experimental design included two independent variables: five distinct categories of mid-air gestures (physical, symbolic, metaphorical, abstract, and mixed) and memory set sizes ranging from two to nine items. The findings revealed that working memory capacities for different types of gestures fluctuate between three and five items, with symbolic gestures presenting the most negligible cognitive load for recall and abstract gestures proving to be the most challenging. The outcomes of this research have significant implications for both the design of individual gestures and the overarching design of gesture interaction systems, serving as a valuable reference for future endeavors in HCI design optimization.

Barrier function–based adaptive STSMC for steering feel feedback of SBW vehicle with uncertainty

As an essential component of human–computer interaction in steer-by-wire (SBW) vehicles, the steering feel feedback system provides drivers with accurate steering feel by precisely controlling the output torque of the steering feel motor. Meanwhile, the existence of uncertainties makes the accurate control of steering feel feedback torque become an acknowledged challenging issue. This article proposes an adaptive super-torque sliding mode control strategy based on barrier functions for the steering feel feedback system of SBW vehicles with highly complex dynamics in the presence of uncertainties. First, to achieve intelligent modeling under the influence of uncertainty factors, a dynamic system with unknown uncertainty based on adaptive fuzzy logic system (FLS) is proposed. In the framework of this model, the uncertainty factors introduced by unknown external disturbance, parameter uncertainty, and model couplings are considered as a lumped uncertainty function. Furthermore, the membership functions of the FLS are dynamically adjusted based on nearest neighbor clustering, aiming to enhance the modeling accuracy beyond traditional fuzzy modeling. To further offset the system uncertainty and FLS approximation error, an adaptive super-twisting sliding mode control (STSMC) method based on barrier function is proposed. This algorithm retains the advantages of traditional sliding mode control (SMC) methods in dealing with systems with uncertainty and eliminates the need for a priori knowledge of the upper bound of uncertainties by introducing the adaptive law based on barrier function, avoiding the use of complex identification algorithms. Moreover, it eliminates the gain of overestimation and significantly reduces chattering phenomenon in traditional sliding mode control. Finally, based on Lyapunov stability theory, this article proves the proposed system can achieve convergence within a finite time. Compared with STSMC, traditional SMC and proportional–integral–derivative (PID) control, the results verify the effectiveness of the proposed control method. The proposed strategy provides a new solution to reduce the steering feel feedback torque error under the influence of uncertain factors.

Sustainable and high performance MXene hydrogel with interlocked structure for machine learning-facilitated human-interactive sensing

Wearable epidermal electronics fabricated from MXene-based conductive hydrogels have attracted considerable interest because of their promising prospects in real-time health monitoring and human–computer interaction sensing. However, the development of a sustainable hydrogel with exceptional mechanical toughness, high conductivity, and self-adhesion is crucial to reliably capture signals through polymer network designs that establish robust interactions between MXene nanosheets and network substrates while minimizing electronic waste generated by discarded sensors. This study presents the fabrication of a sustainable conductive MXene-based dual-network hydrogel through skillful assembly of MXene nanosheets, β-cyclodextrin with azobenzene-modified polyacrylic acid chains, and poly (vinyl alcohol) polymer networks. The hydrogel demonstrates exceptional mechanical properties (strength at breaking ∼ 625 kPa, elongation at break ∼ 630 %) attributed to the presence of numerous hydrogen bonds and host–guest interactions, along with superior electrical conductivity (23.65 S/cm) and self-adhesion. The hydrogel demonstrates exceptional sensitivity across a wide range of strains, enabling the detection of various human movements and achieving successful applications in handwriting recognition and sign language translation. Under acidic condition and UV irradiation, the disassembly of AZO and β-CD as well as the untangling of PVA chains entanglement facilitate the release and recycling of MXene. This study presents a technological platform that holds promise for developing conductive hydrogels with enhanced sensing capabilities, catering to the requirements of stretchable electronic skin and intelligent robotic systems.

Design and Implementation of a Shipborne Echo Sounder Simulator Based on a Seabed Echo Scattering and Noise Model

The Manila Amendment 2010 to the STCW International Convention has made clear requirements for seafarers to use the navigation simulation system for training. A shipborne echo sounder is an important navigation aid equipment necessary for a ship’s bridge. Proper use of this equipment can effectively prevent ship grounding accidents. Given the lack of research on simulating different seabed substrate echoes within echo sounder simulations, this paper proposes an algorithm for generating echoes and clutter from various seabed substrates, based on the Jackson model and noise model. Using the seabed echo generation algorithm, the bathymetric data and seabed echo under the influence of ship rolling are generated, the seabed echo simulation of the sounder under the influence of four different grazing angles and six different substrates is realized. Clutter images, including random noise, bubble interference, co-frequency interference, and fish school interference, are also simulated. A typical ship echo sounder simulator is designed and developed. The echo sounder simulator developed in this paper has high realism in seabed echoes and clutter simulation, complete functions, and friendly human–computer interaction. The system has been used by college students and crews, with satisfactory results, which can effectively meet the needs of actual training of seafarers.

A Review Paper on Human Computer Interaction

The development of computer technology has materialized into the term of human computer interaction. Research studies in human computer interaction studied young age groups that are educated and technically inclined. This paper will study the mental model in Human Computer Interaction that could be useful in some area. There are a number of types of this review paper on the Internet. One will lay out the current approach, result, trends, in the human computer interaction and the other approach will look into research construction that have been around long before but are now forgotten. In this paper, the author is also looking at the landscape in the emotional intelligence of a user so that they are more user like, fidelity prototyping. The development and design of an automated system that performs this task is still being undertaken. The notion of human-computer interaction is conceived as a result of the evolution of the development criteria of computer technology. The next generation of individuals (those at a younger age cohort) in which is termed an educated, technical, or at least knowledgeable individual will also be the focus of individual (tech) research experiments in human-computer interaction. Human-computer interaction (HCI) refers to both technical and human behaving aspects. The central tenets of practical studies in human-computer interaction are to reveal implicit knowledge about human behaviour and human interaction with technology. Resilience is set of behaving contextually taken collectively reacts to challenges. The term resilience has been applied to virtually everything from the economy, real estate, sports, psychology, business, and education, and the list could and can go on. Resilience is mainly comprised or made of a number of different abilities and skills as a way to be able to relate effectively to others, build relationships, self-efficacy, optimism, and self-awareness, creating meaning from other experiences form these are essential combination of competencies. In this manner people should use this in order to optimize the value of the resilience of an organization. This becomes important resourcefulness knowledge of the help available to increasing capacity to deal with the challenges facing change, overcoming (wasting time) relative to everything that could be done alternatively.

Hierarchical MPC-based authority allocation strategy for human–machine shared vehicles considering human–machine conflict

The uncertainty of driver behavior is an important factor affecting the safety of human–machine co-driving vehicles. Traditional rule-based models often fail to capture the nonlinear and complex characteristics of human steering behavior. To overcome this, we propose a data network-driven approach utilizing gated recurrent unit (GRU) neural networks to accurately predict driver steering behavior. The GRU-based driver model is integrated with vehicle dynamics to construct a control-oriented driver–vehicle model. Considering that human–machine conflict may cause vehicle instability, an exponential function combining proportional–integral–derivative is proposed to quantify the human–machine conflict level based on steering difference. To reasonably allocate human–computer permissions based on human–machine interaction, a hierarchical authority allocation framework is proposed. The upper layer provides a reference authority allocation via an exponential function, while the lower layer employs a real-time model predictive control (MPC) optimizer to track this reference, ensuring optimal vehicle path tracking and stability. The proposed system’s effectiveness is validated through driver-in-the-loop testing, demonstrating significant improvements in safety and performance. The results show that in the human–machine conflict scenario, the proposed authority allocation strategy can still ensure the path tracking and safety of the vehicle.

Supporting Human–Robot Interaction in Manufacturing with Augmented Reality and Effective Human–Computer Interaction: A Review and Framework

The integration of Augmented Reality (AR) into Human–Robot Interaction (HRI) represents a significant advancement in collaborative technologies. This paper provides a comprehensive review of AR applications within HRI with a focus on manufacturing, emphasizing their role in enhancing collaboration, trust, and safety. By aggregating findings from numerous studies, this research highlights key challenges, including the need for improved Situational Awareness, enhanced safety, and more effective communication between humans and robots. A framework developed from the literature is presented, detailing the critical elements of AR necessary for advancing HRI. The framework outlines effective methods for continuously evaluating AR systems for HRI. The framework is supported with the help of two case studies and another ongoing research endeavor presented in this paper. This structured approach focuses on enhancing collaboration and safety, with a strong emphasis on integrating best practices from Human–Computer Interaction (HCI) centered around user experience and design.

DIRA: A model of the user interface

The user interface is a central concept in human–computer interaction, but peculiarly fuzzy. This affects discussions of the fundamentals of our discipline and the positioning of our work. We propose a model of the user interface that consists of four elements: Devices, Interaction Techniques, Representations, and Assemblies (DIRA). We explain their roles in the user interface and discuss some associated concerns about evaluation and design. We then show how to use the model to describe the elements of user interfaces (with examples that include a menu, a fisheye interface, and notifications) and to analyze the central characteristics of user interface paradigms (including tangible user interfaces and mixed reality). Finally, we discuss how describing user interfaces with the model can drive their design and evaluation.

Enhancing human computer interaction with coot optimization and deep learning for multi language identification

Human-Computer Interaction (HCI) is a multidisciplinary field focused on designing and utilizing computer technology, underlining the interaction interface between computers and humans. HCI aims to generate systems that allow consumers to relate to computers effectively, efficiently, and pleasantly. Multiple Spoken Language Identification (SLI) for HCI (MSLI for HCI) denotes the ability of a computer system to recognize and distinguish various spoken languages to enable more complete and handy interactions among consumers and technology. SLI utilizing deep learning (DL) involves using artificial neural networks (ANNs), a subset of DL models, to automatically detect and recognize the language spoken in an audio signal. DL techniques, particularly neural networks (NNs), have succeeded in various pattern detection tasks, including speech and language processing. This paper develops a novel Coot Optimizer Algorithm with a DL-Driven Multiple SLI and Detection (COADL-MSLID) technique for HCI applications. The COADL-MSLID approach aims to detect multiple spoken languages from the input audio regardless of gender, speaking style, and age. In the COADL-MSLID technique, the audio files are transformed into spectrogram images as a primary step. Besides, the COADL-MSLID technique employs the SqueezeNet model to produce feature vectors, and the COA is applied to the hyperparameter range of the SqueezeNet method. The COADL-MSLID technique exploits the SLID process’s convolutional autoencoder (CAE) model. To underline the importance of the COADL-MSLID technique, a series of experiments were conducted on the benchmark dataset. The experimentation validation of the COADL-MSLID technique exhibits a greater accuracy result of 98.33% over other techniques.

Icon similarity model based on cognition and deep learning

Human-computer cooperation guided by natural interaction, intelligent interaction, and human–computer integration is gradually becoming a new trend in human–computer interfaces. An icon is an indispensable pictographic symbol in an interface that can convey pivotal semantics between humans and computers. Research on similar icons’ cognition in humans and the discrimination of computers can reduce misunderstandings and facilitate transparent cooperation. Therefore, this research focuses on images of icons, extracted contours, and four features, including the curvature, proportion, orientation, and line of the contour, step by step. By manipulating the feature value change to obtain 360 similar icons, a cognitive experiment was conducted with 25 participants to explore the boundary values of the feature dimensions that cause different levels of similarity. Its boundary values were applied to deep learning to train a discrimination algorithm model that included 1500 similar icons. This dataset was used to train a Siamese neural network using a 16-layer network branch of a visual geometry group. The training process used stochastic gradient descent. This method of combining human cognition and deep learning technology is meaningful for establishing a consensus on icon semantics, including content and emotions, by outputting similarity levels and values. Taking icon similarity discrimination as an example, this study explored the analysis and simulation methods of computer vision for human visual cognition. The accuracy evaluated is 90.82%. The precision was evaluated as 90% for high, 80.65% for medium, and 97.30% for low. Recall was evaluated as 100% for high, 89.29% for medium, and 83.72% for low. It has been verified that it can compensate for fuzzy cognition in humans and enable computers to cooperate efficiently.