Human-computer Interaction Techniques Research Articles

Human-Computer Interaction Techniques for Explainable Artificial Intelligence Systems

As Artificial Intelligence (AI) systems become more widespread, there is a growing need for transparency to ensure human understanding and oversight. This is where Explainable AI (XAI) comes in to make AI systems more transparent and interpretable. However, developing adequate explanations is still an open research problem. Human-Computer Interaction (HCI) is significant in designing interfaces for explainable AI. This article reviews the HCI techniques that can be used for solvable AI systems. The literature was explored with a focus on papers at the intersection of HCI and XAI. Essential techniques include interactive visualizations, natural language explanations, conversational agents, mixed-initiative systems, and model introspection methods while Explainable AI presents opportunities to improve system transparency, it also comes with risks, especially if the explanations need to be designed carefully. To ensure that explanations are tailored for diverse users, contexts, and AI applications, HCI principles and participatory design approaches can be utilized. Therefore, this article concludes with recommendations for developing human-centred XAI systems, which can be achieved through interdisciplinary collaboration between HCI and AI. As Artificial Intelligence (AI) systems become more common in our daily lives, the need for transparency in these systems is becoming increasingly important. Ensuring that humans clearly understand how AI systems work and can oversee their functioning is crucial. This is where the concept of Explainable AI (XAI) comes in to make AI systems more transparent and interpretable. However, developing adequate explanations for AI systems is still an open research problem. In this context, Human-Computer Interaction (HCI) is significant in designing interfaces for explainable AI. By integrating HCI principles, we can create systems humans understand and operate more efficiently. This article reviews the HCI techniques that can be used for solvable AI systems. The literature was explored with a focus on papers at the intersection of HCI and XAI. The essential methods identified include interactive visualizations, natural language explanations, conversational agents, mixed-initiative systems, and model introspection methods. Each of these techniques has unique advantages and can be used to provide explanations for different types of AI systems. While Explainable AI presents opportunities to improve system transparency, it also comes with risks, especially if the explanations need to be designed carefully. There is a risk of oversimplification, leading to misunderstanding or mistrust of the AI system. It is essential to employ HCI principles and participatory design approaches to ensure that explanations are tailored for diverse users, contexts, and AI applications. By developing human-centred XAI systems, we can ensure that AI systems are transparent, interpretable, and trustworthy. This can be achieved through interdisciplinary collaboration between HCI and AI. The recommendations in this article provide a starting point for designing such systems. In essence, XAI presents a significant opportunity to improve the transparency of AI systems, but it requires careful design and implementation to be effective.

A case study of virtual CNC machine tool assembly and human-computer interaction techniques

When it comes to the virtual machine tool assembly process, there are problems such as assembly errors. Based on the general idea of correct assembly, it is proposed to innovate the assembly process by utilizing the enclosing box and separating axis theorem and GJK algorithm, so as to monitor the state of the detection process, which verifies the effective feasibility of the monitoring method and provides a new idea for the assembly process of virtual CNC machine tools.

SMR: Spatial-Guided Model-Based Regression for 3D Hand Pose and Mesh Reconstruction

3D hand reconstruction is an important technique for human-computer interaction. Interactive experience depends on the accuracy, efficiency, and robustness of the algorithm. Therefore, in this paper, we first propose a balanced framework called spatial-aware regression (SAR) to achieve precise and fast reconstruction. SAR can bridge convolutional networks and graph-structure networks more effectively than existing frameworks to fully exploit extracted spatial information using a novel spatial-aware initial graph building module. In addition, SAR uses adaptive-GCN to make keypoints interact efficiently and effectively; and regresses 2.5D belief maps to characterize uncertainty. SAR is highly flexible because it can predict an arbitrary number of keypoints and apply pose-guided refinement for coarse to fine regression. To produce more rational results for challenging cases and mitigate 3D label reliance, we also propose a more robust model-based framework called spatial-guided model-based regression (SMR) that is based on SAR. There are two critical designs of SMR: 1) it uses SAR to enhance the features with pose information to help the regression of hand model parameters; and 2) it regresses parameters in a spatially aware manner that is similar to SAR. Experiments demonstrate that the proposed frameworks surpass existing fully-supervised approaches on the FreiHAND, HO-3D, RHD, and STB datasets. Also, the performances of the proposed frameworks under weakly/self-supervised settings outperform other competitors. Meanwhile, the proposed frameworks are accurate and efficient.

Application of Human-Computer Interaction and Digital Image Processing Technology in Virtual Simulation Game Design

Abstract This paper first analyzes the design process, image imaging, and view reconstruction of virtual simulation game scenes and discusses the key technologies used in virtual simulation games. Then, the human-computer interaction techniques of gesture, pose, voice, and data interaction are combined to design a multi-touch detection algorithm based on human-computer interaction. Finally, the edge detection algorithm is combined with the edge detection algorithm to detect the edge of the image, and after the displacement conversion and extraction, the processing process of the digital image is completed. After the performance test, the usability of pose interaction, the accuracy of speech recognition and the timeliness of data interaction of the virtual simulation game are between 0.8 and 1.0, and the overall human-computer interaction capability is highly evaluated.

PD-FMCW: Push the Limit of Device-Free Acoustic Sensing Using Phase Difference in FMCW

Device-free acoustic sensing has obsessed with renovating human-computer interaction techniques for all-sized mobile devices in various applications. Recent advances have explored sound signals in different methods to achieve highly accurate and efficient tracking and recognition. However, accuracies of most approaches remain bottlenecked by the limited sampling rate and narrow bandwidth, leading to restrictions and inconvenience in applications. To bridge over the aforementioned daunting barriers, we propose PD-FMCW, a novel ultrasound-based device-free tracking scheme that can distinctly improve the resolution of fine-grained sensing to submillimetre level. In its heart lies an original Phase Difference based approach leveraging the Frequency-Modulated Continuous Wave to derive the reflected time delay, thus precisely inferring absolute distance, catering to interaction needs of tinier perception with lower delay. The distance resolution of PD-FMCW is only related to the speed of actions and chirp duration. We implement a prototype with effective denoising methods all in the time domain on smartphones. The evaluation results show that PD-FMCW achieves accuracies of 2.5 mm, 3.6 mm, and 2.1 mm in distance change, path length change, and trajectory tracking error respectively. PD-FMCW is also valid in recognizing 2 mm or even tinier micro-movements, which paves the way for more delicate sensing work.

Superstretchable and multifunctional ionogel for facile fabricating flexible strain sensors

With the rapid development of “Internet of Things” and human-computer interaction techniques, it is essential and urgent to develop facile and scalable fabrication platforms for stretchable flexible sensor. Herein, we report a facile strategy of using the green choline chloride–acrylamide deep eutectic solvent (CC-AM DES) to guide the in-situ ring-opening polymerization of α-lipoic acid (LA), leading to the successful development of a stretchable ionogel material. The as-prepared ionogel from CC-AM DES system exhibits multifunctional merits including the super stretchability (>9000%), 100% UV-blocking ability, tunable adhesiveness (29–414 kPa), high ionic conductivity (4.45 × 10−4 S/cm), and ideal anti-freezing (–27 °C). In addition, this outstanding ionogel can be readily coated on various material substrates with designable shapes and patterns. Owning to these promising properties and performances, a scalable flexible strain sensor is assembled from the ionogel and exhibits stable resistance variations (R/R0) towards multiple external mechanical stimulus. This study provides a green, cost effective, and scalable strategy to fabricate ionogel materials and multifunctional flexible strain sensors, showing a great potential in the fast-emerging highly stretchable wearable/flexible electronics.

The rising trend of Metaverse in education: challenges, opportunities, and ethical considerations.

Metaverse is invading the educational sector and will change human-computer interaction techniques. Prominent technology executives are developing novel ways to turn the Metaverse into a learning environment, considering the rapid growth of technology. Since the COVID-19 outbreak, people have grown accustomed to teleworking, telemedicine, and numerous other forms of distance interaction. Recently, the Metaverse has been the focus of many educators. With Facebook's statement that it was rebranding and promoting itself as Meta, this field saw a surge in interest in the areas of computer science and education. There is a literature gap in studying the Metaverse's role in education. This article is a systematic review following the PRISMA framework that reviews the role of the Metaverse in education to shrink the literature gap. It presents various educational uses to aid future research in this field. Additionally, it demonstrates how enabling technologies like extended reality (XR) and the internet of everything (IoE) will significantly impact educational services in the Metaverses of the future of teaching and learning. The article also outlines key challenges, ethical issues, and potential threats to using the Metaverse for education to offer a road map for future research that will investigate how the Metaverse will improve learning and teaching experiences.

Leveraging metaheuristics with artificial intelligence for customer churn prediction in telecom industries

<abstract> <p>Customer churn prediction (CCP) is among the greatest challenges faced in the telecommunication sector. With progress in the fields of machine learning (ML) and artificial intelligence (AI), the possibility of CCP has dramatically increased. Therefore, this study presents an artificial intelligence with Jaya optimization algorithm based churn prediction for data exploration (AIJOA-CPDE) technique for human-computer interaction (HCI) application. The major aim of the AIJOA-CPDE technique is the determination of churned and non-churned customers. In the AIJOA-CPDE technique, an initial stage of feature selection using the JOA named the JOA-FS technique is presented to choose feature subsets. For churn prediction, the AIJOA-CPDE technique employs a bidirectional long short-term memory (BDLSTM) model. Lastly, the chicken swarm optimization (CSO) algorithm is enforced as a hyperparameter optimizer of the BDLSTM model. A detailed experimental validation of the AIJOA-CPDE technique ensured its superior performance over other existing approaches.</p> </abstract>

Data Analytics and Visualization Application for Asset Health Monitoring

Much of the research on predictive maintenance has focused on statistical and machine learning techniques, while there has been significantly less focus on the human computer interaction or visualization aspects of PHM. Human computer interaction and visualization techniques can quickly help identify interesting data sub-domains, both time periods and sensors, provided the data can be queried, retrieved and displayed in a timely manner. Augmenting visualization and interaction with a visual, aggregative fleet-based query system adds a further dimension, highlighting the ability of the fleet to carry out its mission. In the age of big data, effectively visualizing data from an asset with a multitude of sensors can be difficult, and that difficulty is compounded as you add additional assets.
 In this paper, we propose a scalable framework that is capable of visualizing past, current, and prognosticated health from the individual sensor up to the fleet or group level. In addition to viewing near real time sensor data, maintenance logs, fault information, and data aggregations will be merged with the sensor data to make the analysis and visualizations more valuable. This framework is scalable in regard to how much data can be collected, stored, and processed, as well as to the different organizational levels within a fleet of assets. The framework is built as a web-application primarily using the following visualizations: a collapsible tree graph for asset information; 2D charts for temporal sensor data, fault data, and maintenance data; and 3D digital twins of critical components. 
 The current dataset used to build and demonstrate the capability of the web-application has data from over 3000 vehicles and contains approximately 9TB of data. Vehicle information such as model, make, sub-component, and fleet organization are presented in a configurable, collapsible tree structure. This allows the user to visualize the fleet and to select the asset and sensor combinations needed to display the temporal sensor data that will answer the question at hand. Information regarding each vehicle’s health status is displayed then aggregated and displayed for each of the higher tree nodes. A 3D digital twin is also available to highlight sensor locations and current health status of the component. These component models can be viewed and manipulated with and without a virtual reality headset. As health status monitoring for asset subcomponents are developed, they can be added to the system, allowing for complete health status reporting.

Toward Robust Device-Free Gesture Recognition Based on Intrinsic Spectrogram of mmWave Signals

Device-free gesture recognition is a potential noncontact human–computer interaction technique. It leverages the unique influence of the conducted gesture on surrounding wireless signals to accomplish gesture recognition. Existing methods usually leverage doppler spectrogram of the influenced wireless signals to characterize the motion pattern of gestures. These methods have achieved satisfactory accuracy when the gestures are conducted in a relatively fixed location, direction, and speed. However, when gestures are conducted in a different scenario, the recognition accuracy will drop dramatically. In this article, we try to solve this issue by characterizing the gesture motion pattern using a novel robust intrinsic spectrogram, which is independent of the conducted scenario. Specifically, we create a virtual coordinate system in which the coordinates of a gesture trajectory remain unchanged no matter where and how the gesture is conducted. Then, we design a coordinate transformation method to transform the raw doppler spectrogram into the robust intrinsic spectrogram to characterize the intrinsic motion pattern of the gesture. We further feed the intrinsic spectrogram into a deep network to realize gesture recognition. Extensive evaluations on a 77-GHz mmWave testbed show that the proposed method could achieve an average recognize accuracy of 88.4% with ten types of gestures.

A Systematic Review of Human–Computer Interaction (HCI) Research in Medical and Other Engineering Fields

This article provides a systematic review of research related to Human–Computer Interaction techniques supporting training and learning in various domains including medicine, healthcare, and engineering. The focus is on HCI techniques involving Extended Reality (XR) technology which encompasses Virtual Reality, Augmented Reality, and Mixed Reality. HCI-based research is assuming more importance with the rapid adoption of XR tools and techniques in various training and learning contexts including education. There are many challenges in the adoption of HCI approaches, which results in a need to have a comprehensive and systematic review of such HCI methods in various domains. This article addresses this need by providing a systematic literature review of a cross-section of HCI approaches involving proposed so far. The PRISMA-guided search strategy identified 1156 articles for abstract review. Irrelevant abstracts were discarded. The whole body of each article was reviewed for the remaining articles, and those that were not linked to the scope of our specific issue were also eliminated. Following the application of inclusion/exclusion criteria, 69 publications were chosen for review. This article has been divided into the following sections: Introduction; Research methodology; Literature review; Threats of validity; Future research and Conclusion. Detailed classifications (pertaining to HCI criteria and concepts, such as affordance; training, and learning techniques) have also been included based on different parameters based on the analysis of research techniques adopted by various investigators. The article concludes with a discussion of the key challenges for this HCI area along with future research directions. A review of the research outcomes from these publications underscores the potential for greater success when such HCI-based approaches are adopted during such 3D-based training interactions. Such a higher degree of success may be due to the emphasis on the design of user-friendly (and user-centric) training environments, interactions, and processes that positively impact the cognitive abilities of users and their respective learning/training experiences. We discovered data validating XR-HCI as an ascending method that brings a new paradigm by enhancing skills and safety while reducing costs and learning time through replies to three exploratory study questions. We believe that the findings of this study will aid academics in developing new research avenues that will assist XR-HCI applications to mature and become more widely adopted.

Dynamic hyperparameter tuning-based path tracking control for robotic rollers working on earth-rock dam under complex construction conditions

Robotic roller technology is important for ensuring the compaction quality and construction efficiency of earth-rock dams, and its key challenge is related to path tracking control. The current easy-to-use and effective proportion–integration–differentiation (PID) path tracking controller requires time-consuming and labor-intensive hyperparameter tuning, making its application in complex construction environments with a set of fixed hyperparameters, such as transition, misalignment, and position drift, difficult. This paper describes the TSABFA-PID path tracking controller based on hybrid tunicate swarm algorithm (TSA) and bacterial foraging algorithm (BFA), and modified kinematic model. The proposed controller can dynamically self-tune hyperparameters to adapt to complex construction condition by optimizing the objective function. The proposed controller achieved better results in both simulation environment and on-site tests than state-of-the-art methods. In the future, the deep reinforcement learning and human-computer interaction techniques will be integrated into the proposed controller to improve the compaction performance.

Biocompatible, stretchable, and compressible cellulose/MXene hydrogel for strain sensor and electromagnetic interference shielding

ABSTRACT With the booming growth of flexible human-computer interactions and telecommunications, it is desirable to fabricate a high-sensitivity strain sensor to monitor human movement and develop an electromagnetic interference (EMI) shielding materials to protect modern microelectronics and humans from electromagnetic damage. Herein, the biocompatible, stretchable, and compressible cellulose/MXene hydrogel has been prepared to first realize the dual-functional applications in strain sensor and EMI shielding. Two-dimensional MXene nanosheets serve as conductive sensing materials and fillers to construct conductive networks. The hydrogen bond interaction between them enhances the mechanical property of hydrogel. Benefiting from the excellent electrical conductivity and good mechanical property of the cellulose/MXene hydrogel, the resultant strain sensor displays excellent tensile strain (~144.4%), high sensitivity (gauge factor ~ -6.97), adjustable detection range (0-68.7%), fast response time (~100 ms), and great stability (~1000 cycles). It can monitor human motion, including pulse beating, speech recognition, writing sensing and pressure distribution induction of 4 × 4 sensor array platform. Moreover, the cellulose/MXene hydrogel presents an absorption-predominant EMI shielding performance in the frequency of X-band. This work provides a new inspiration for the development of multifunctional hydrogel in artificial intelligence, human-computer interaction, and EMI shielding technique.

Education for Sustainability and the Development of Smart Healthcare Systems: An Exploratory Study

The development of smart healthcare systems that are accurate and efficient may be used to improve the health and well-being of different age groups. These systems should incorporate a human-centered design approach to ensure that products and services meet user needs and that systems are user-friendly. In this study, an "education for sustainability" perspective is used in tandem with human-factors engineering and human-computer interaction techniques to achieve a creative dimensional design within a workshop setting. Workshop settings help learners transcend the limitations of traditional classroom education by encouraging them to integrate their daily needs and develop feasible healthcare solutions. Using an iterative process, proposed designs are repeatedly validated and prototypes are continuously improved. We hope that this article provides educators with a better sustainable educational perspective that they may use to construct accurate and efficient smart healthcare systems that meet the needs of users.

FREE GUY- A Hand Gesture Game Based on ML

Human Computer Interaction techniques became a bottleneck within the effective utilization of the obtainable info flow. The development of user interfaces affects changes in the Human Computer Interaction (HCI). The naturalness and intuitiveness of the hand gesture has been a major motivating factor for researchers in the HCI field to invest their efforts in researching and developing the most promising means of Human-Computer Interaction. This paper will let you know about the creation of hand gesture recognition program and implementation of its techniques in the computer game by controlling the movements of character introduced. The machine learning part is accomplished by the libraries like Tensorflow and OpenCV for setting up the detection, recognition and image processing techniques.

The optimization algorithm for application in directional drilling trajectories of energy field

Recently, the directional drilling technology (DDT) has been extensive used in many energy field such as coal geological exploration, gas drainage and artificial freezing engineering etc. The theoretical model of directional drilling trajectories is insufficient leading to drilling trajectories control is almost uncertain. In the present study, a theoretical model of directional drilling trajectories is proposed, which is simplified as a combination of many alternants constant curvatures smoothing quasi-helical segments. The essential condition of nonlinear piecewise systems is proved by maximal principle. Multimodal objective function is figure out by optimization algorithm. The directional drilling trajectories software is programmed, and the optimum trajectory for directional drilling is designed. The maximum directional drilling deviation on the vertical direction is 0.38 m, and horizontal direction is 0.63 m. The optimization algorithm is more intuitive and reliable than repetition test and traditional human–computer interaction techniques.

Stretchable, self-adhesive, conductive, anti-freezing sodium polyacrylate-based composite hydrogels for wearable flexible strain sensors

The vigorous development of flexible sensors, wearable electronic devices, and human-computer interaction techniques has promoted the construction of flexible, stretchable, and conductive hydrogels. However, due to the lack of self-adhesion and environmental adaptability, these hydrogels still face huge challenges in accurately and stably monitoring human motions as flexible sensors. In this paper, a multifunctional hydrogel of poly(sodium polyacrylate-dopamine based methylacrylamide)/polyvinyl alcohol (i.e., P(AAS-DAAM)/PVA) with the properties of stretchable, self-adhesive, conductive, and anti-freezing was synthetized by introducing methacrylic acid modified dopamine into sodium polyacrylate crosslinked network, and combine with a physical cross-linked network of the polyvinyl alcohol. The hydrogel exhibits excellent stretchability (>1000%), the higher conductivity (0.038 ± 0.003 S·cm−1), good frost resistance (−30°C) and excellent adhesion (419.3 ± 5.95 N/m) and remarkable biocompatibility (relative cell viability up to 132 ± 2.63%). The results indicate that P(AAS-DAAM)/PVA hydrogel can be used as a flexible sensor. Moreover, P(AAS-DAAM)/PVA-based strain sensor not only can stably and repeatedly monitor the changes of different strains (50% ~ 200%) with high sensitivity (GF = 1.65), but also stable and accurate monitoring capabilities for large joint bending (fingers, wrists, elbows, knees) or small physical (swallowing, speaking) activities. This hydrogel preparation strategy strongly promotes the development of new multifunctional hydrogels and hydrogel-based flexible sensors and other equipment to a new step.

Magic-hand: Turn a smartwatch into a mouse

We present the design, implementation and evaluation of a smartwatch-based, freehand human–computer interaction system, called Magic-Hand, which consists of two modules: the operation detection module and the motion tracking module. In the operation detection module, five gestures are selected to represent typical operations including right-click, left-click and double-click, and we carefully design the detection algorithm to identify gestures even when the hand is in motion. In the motion tracking module, we map the movement of the hand to the screen cursor movement with the Sigmoid function. We verify the performance of Magic-Hand by comparing it with two existing human–computer interaction techniques: the mouse and the AirMouse. The experimental results show that the gesture recognition in the operation detection module can reach an accuracy of 95%, and the motion tracking is comparable with the benchmarks. Our work facilitates the use of the smartwatch as an effective interaction device in ubiquitous display environments.

PENGGALIAN KEBUTUHAN MENGGUNAKAN METODE PERSONA PADA SISTEM PENILAIAN LMS

The development of a software is very necessary to understand the users/users of the system. The aspect of user convenience in using the system is a very important aspect so that the system that has been created is in accordance with the purpose of its manufacture and according to user needs. Persona is a technique of Human Computer Interaction by collecting information from users as a reference in understanding the needs of users. The purpose of this study is to apply the persona technique by adjusting the validation process to the user personas of our system users. The method used in this research is the Human Computer Interaction technique. User persona is used as a reference so that a system is made based on user needs so that the purpose of making the system can be met. The result of this research is to use the user persona technique in the elicitation of needs which will facilitate understanding related to the development of a system.Keywords: Human Computer Interaction, Persona, Elicitation of Needs

Assessment of Requirements Engineering and Human-Computer Interaction Techniques in the Production Process of Business Intelligence Applications

Assessment of Requirements Engineering and Human-Computer Interaction Techniques in the Production Process of Business Intelligence Applications