Human-computer Interaction Process Research Articles

Establishing a user demand hierarchy model driven by a mental model for complex operating systems

In complex human-computer interactions, issues such as task failures, system malfunctions, and frequent accidents caused by user errors are common. Therefore, it is necessary to study complex system interactions to enhance overall efficiency. This study focuses on the task interface of a digital twin system for computer numerical control (CNC) machine tools and examines the relationship between the user mental model and interface design elements. Key mental information is obtained through questionnaires and interviews, forming the basis for establishing a user mental model. High-frequency information words are extracted, experimental samples are designed, and importance rating surveys are conducted. Quantitative analysis methods, including factor analysis and weight calculation, are utilized to analyze the needs of target users. Consequently, a user demand hierarchy model is constructed. This approach aims to effectively reduce user errors in the human-computer interaction process within complex systems and enhance cognitive efficiency.

Long Short-Term Memory (LSTM) model for Indian sign language recognition

The human-computer interaction process is a vital task in attaining artificial intelligence, especially for a person suffering from hearing or speaking disabilities. Recognizing actions more traditionally known as sign language is a common way for them to interact. Computer vision and Deep learning models are capable of understanding these actions and can simulate them to build up a sustainable learning process. This sign language mechanism will be helpful for both the persons with disabilities and the machines to unbound the gap to achieve intelligence. Therefore, in the proposed work, a real-time sign language system is introduced that is capable of identifying numbers ranging from 0 to 9. The database is acquired from the 8 different subjects respectively and processed to achieve approximately 200k amount of data. Further, a deep learning model named LSTM is used for sign recognition. The results were compared with different approaches and on distinct databases proving the supremacy of the proposed work with 91.50% accuracy. Collection of daily life useful signs and further improving the efficiency of the LSTM model is the research direction for future work. The code and data will be available at https://github.com/rahuln2002/Sign-Language-Recognition-using-LSTM-model.

Capabilities Comparison Of The Augmented Reality Application Development Frameworks On The Android Platform

In the implementation of augmented reality, the natural process of human-computer interaction continues to be a challenge, specifically to reduce the complexity in the effort of using and providing comfortability. Therefore, it is necessary to search for methods and tools to simplify the complex process of building Augmented Reality applications. Vuforia is a multi-platform that has long been used as an Augmented Reality application development framework. Android and iOS platform frameworks are the new alternatives following the development of cellular technology. Furthermore, a comparative capability was conducted between Vuforia and ARCore on the Android platform. The general performance and the ability to understand the environment include working in horizontal and vertical planes, the ability to work based on lighting conditions, and the distance of the camera during the tests. The results showed that ARCore is superior to Vuforia in almost all testing metrics. However, in overexposed or too dim lighting and at very close surfaces, Vuforia is slightly superior but not essential. Therefore, it can be concluded that ARCore's capability is better than Vuforia.

Computer Vision-Based Pier Settlement Displacement Measurement of a Multispan Continuous Concrete Highway Bridge under Complex Construction Environments

Various concrete bridges have been built across oceans, valleys, and mountains; however, the settlement displacement of bridge piers caused by environmental changes or self-weight during construction phases often leads to uneven stresses, cracking, and eventual collapse. To address the labor-intensive and high-cost issues of pier displacement monitoring using contact-type sensors, this paper proposes an automatic vision-based method for measuring pier settlement displacement under complex construction environments, such as complex image backgrounds, varying ambient light, and camera movement. In the proposed method, a deep learning network was first employed to eliminate the adverse effect of complex image backgrounds and varying ambient light on the accuracy of target detection; then, an adaptive displacement extraction algorithm without a human-computer interaction process was developed to automatically extract the center coordinates of targets attaching to the bridge piers and reference platform; finally, the pier settlement displacement was calculated by using the relative displacements obtained by a dual camera system to eliminate the measurement error caused by camera translation and rotation movements. Laboratory tests of a cantilever beam and field tests of a continuous multispan concrete girder highway bridge under construction have successfully validated the effectiveness and robustness of the developed methodology. The results obtained in this paper can provide some insights for engineers in applying computer vision technology for the real-time monitoring of bridge displacements.

A REVIEW ON IMPLEMTATION OF VISION GLIDE TECHNIQUE FOR SMOOTH NAVIGATION WITH CAMERA BASED VIRTUAL MOUSE

As artificial intelligence technology has advanced, it has become commonplace to employ hand gesture detection to control virtual objects. The suggested system in this research is a hand gesture-controlled virtual mouse that uses AI algorithms to recognize hand gestures and transform them into mouse movements. People who have trouble using a conventional mouse or keyboard can use the system to provide an alternate interface. The suggested method takes pictures of the user's hand with a camera, which an AI program then utilizes to identify the motions the user is making. Since the development of computer technology, the method for constructing a process of human-computer interaction is advancing. The mouse is one of the best pieces of HCI (Human-Computer Interaction) technology ever created. This article suggests an HCI-based virtual mouse system that makes use of hand movements and computer vision. webcam or built-in camera recordings of gestures that have been subjected to a color segmentation and detection procedure. This system uses a webcam or an integrated camera to capture frames. It then processes the frames to make them trackable, identifies various user motions, and executes mouse functions. In order to make the interaction more effective and dependable, this study suggests a vision-based system to control various mouse behaviors, such as left and right clicking, using hand gestures. This system uses a webcam or an integrated camera to capture frames. It then processes the frames to make them trackable, identifies various user motions, and executes mouse functions. Therefore, the suggested mouse solution removes the need for a device to use a mouse. Therefore, it can be seen that the development of HCI technology is advantageous. Keywords- HCI (Human Computer Interaction), Hand Gesture, Gesture Recognition, OpenCV, Media-pipe, pyAutoGUI.

Inclusive virtual simulation space: Digital technologies and design thinking process

This article demonstrates a case study in research-creation based on the extended reality (XR) placemaking initiative, and reports on the development of two iterations that are part of a digital technologies (DT) project. Various studies with a focus on restorative and relaxation applications have shown positive findings in improving mental wellness and self-awareness. We speculate that combining physical activity and exposure to digital nature can provide additional health benefits compared to restorative activities alone. The first iteration, Mind Space XR, is a virtual reality (VR) project that has the initial goal of supporting users to experience emotion-focused skills training based on mindfulness, emotion regulation and compassion in a low-risk environment. In this iteration of the DT project, we have focused on interaction design aspects and the research-creation process required to develop the information architecture framework of an early prototype. This ongoing research then informs the development of a second iteration, the ‘inclusive virtual simulation space’ – a VR learning simulation initiative that supports users to experience customizable content in an inclusive environment. Combining Australian Curriculum areas from DT and the humanities and social sciences, this study will experiment with the human–computer interaction process with the goal of configuring a modular high-fidelity virtual environment to achieve perceived ease of use by user groups including creators, educators and students. This will lead to a proof-of-concept platform with the flexibility to integrate digital content that permits the co-creation of digital prototypes driven by inclusive virtual and physical placemaking and simulation learning. This educational inclusive virtual simulation project focuses on experimenting with and introducing education technology and XR skills and knowledge to educators and creators, which is broadly applicable to education-based DT contexts.

A Human-Adaptive Model for User Performance and Fatigue Evaluation during Gaze-Tracking Tasks

Eye gaze interfaces are an emerging technology that allows users to control graphical user interfaces (GUIs) simply by looking at them. However, using gaze-controlled GUIs can be a demanding task, resulting in high cognitive and physical load and fatigue. To address these challenges, we propose the concept and model of an adaptive human-assistive human–computer interface (HA-HCI) based on biofeedback. This model enables effective and sustainable use of computer GUIs controlled by physiological signals such as gaze data. The proposed model allows for analytical human performance monitoring and evaluation during human–computer interaction processes based on the damped harmonic oscillator (DHO) model. To test the validity of this model, the authors acquired gaze-tracking data from 12 healthy volunteers playing a gaze-controlled computer game and analyzed it using odd–even statistical analysis. The experimental findings show that the proposed model effectively describes and explains gaze-tracking performance dynamics, including subject variability in performance of GUI control tasks, long-term fatigue, and training effects, as well as short-term recovery of user performance during gaze-tracking-based control tasks. We also analyze the existing HCI and human performance models and develop an extension to the existing physiological models that allows for the development of adaptive user-performance-aware interfaces. The proposed HA-HCI model describes the interaction between a human and a physiological computing system (PCS) from the user performance perspective, incorporating a performance evaluation procedure that interacts with the standard UI components of the PCS and describes how the system should react to loss of productivity (performance). We further demonstrate the applicability of the HA-HCI model by designing an eye-controlled game. We also develop an analytical user performance model based on damped harmonic oscillation that is suitable for describing variability in performance of a PC game based on gaze tracking. The model’s validity is tested using odd–even analysis, which demonstrates strong positive correlation. Individual characteristics of users established by the damped oscillation model can be used for categorization of players under their playing skills and abilities. The experimental findings suggest that players can be categorized as learners, whose damping factor is negative, and fatiguers, whose damping factor is positive. We find a strong positive correlation between amplitude and damping factor, indicating that good starters usually have higher fatigue rates, but slow starters have less fatigue and may even improve their performance during play. The proposed HA-HCI model and analytical user performance models provide a framework for developing an adaptive human-oriented HCI that enables monitoring, analysis, and increased performance of users working with physiological-computing-based user interfaces. The proposed models have potential applications in improving the usability of future human-assistive gaze-controlled interface systems.

A survey on encoding and decoding technology of non-invasive brain-computer interface

A survey on encoding and decoding technology of non-invasive brain-computer interface

Study on Servo Control Method of Eye Movement Intention Based on Reinforcement Learning

Eye movement servo control has the advantage of non-contact and the eye movement is not affected by other limbs. By applying eye-tracking technology to rehabilitation research of people with movement disorders, the interaction between eye-tracking technology and computers will be gradually put into practice, which is equivalent to giving patients a pair of “invisible hands” that can give instructions freely. In this paper, a servo control method based on eye-movement coding technology is proposed to solve the problems of low accuracy, poor reliability, and insufficient human intention initiative in the field of eye-movement servo control of human-computer visual interaction. The human eye movement instruction is processed by coding and integrated with human eye movement intention to realize intelligent servo control of human eye intention. In order to improve the accuracy of eye movement servo control, this paper proposes an eye movement instruction coding processing method, which uses the trained AdaBoost fusion strong classifier algorithm and the landmark detector to detect the real-time algorithm in the video sequence to complete the eye location and blink determination. In this paper, an improved pupil location method based on multi-angle joint detection is proposed. The pupil location is completed by using the information of eye image points, and the digital coding model of eye command action is constructed, which provides an instruction base for eye movement servo control. In order to improve the reliability and initiative of eye-movement servo control, a reinforcement learning method combining eye-movement coded instruction and intention was proposed. By integrating the improved DDPG continuous control strategy reinforcement learning algorithm with eye movement intention instruction, the environmental status can be sensed in the human-computer interaction process and the optimal strategy of eye movement instruction selection can be learned.

A Methodological Study of Human-Computer Interaction: A Review

Abstract: The concept of human-computer interaction has emerged as a result of advancements in computer technology. As emerging technologies in the fields of mobile and cloud computing, as well as the internet of things (IoT), become more and more ingrained in our daily lives, they have started to pose a serious challenge for professionals in human-computer interaction (HCI). These technologies call for greater dedication from HCI experts in terms of systems interface design. A comprehensive study of human-computer interaction (HCI) has been presented in the current study. The design process of human-computer interaction and the current state-of-art of the same is explained in brief along with the applications of HCI. Based on the results of the review, the study recommended combining human-centred design with agile interface design methodologies. It also encouraged future research using a qualitative or quantitative approach to further explore HCI interface design methodologies, with a focus on cloud-based, AR, VR technologies and other organizational information systems.

An HRA Model Based on the Cognitive Stages for a Human-Computer Interface in a Spacecraft Cabin

HRA (Human Reliability Analysis) can be seen as a symmetric problem, as it is mainly reflected in two aspects of failure and success. Human error is the most common accident in industrial systems; furthermore, an astronaut is in a very complex environment, and coupled with weightlessness, it is easy to cause human error. For this reason, this paper took the human-computer interface in a spacecraft cabin as the background, and based on the literature, questionnaire inquiry, and the division of three cognitive processes during the interaction between the astronaut and human, a computer interface was determined. This paper proposed a human reliability influencing factors system of different cognitive phases for the human-computer interface in a spacecraft cabin, a task analysis tree with a symmetry of success and failure, an HRA model with symmetry of failure and success based on cognitive stages, and Game Theory and Fuzzy Center of Gravity Method, and obtained influencing factors weights of three cognitive stages. By simulating an experiment, the trend of error probability curves shows the rationality of the human reliability method. Finally, an example was illustrated, and the analysis process of the example demonstrates that an HRA model with symmetry provides a feasible analysis process and method for the cognitive reliability of the spacecraft cabin human-computer interface interaction. The research achievements in this paper can provide theoretical guidance to improve human error root reason analysis, an analysis basis of how to improve influencing factors level, and provide a HRA method based on cognitive stages for the human-computer interaction process in a spacecraft cabin.

Effects of App Icon Border Form and Interface Background Color Saturation on User Visual Experience and Search Performance

The purpose of this study was to investigate the icon border form and interface background color saturation’s individual and connected impacts on interface interaction. They were two important factors affecting user visual experience and search performance, but previously they have been studied individually. An experiment using eye-tracking, behavior measurement, and subjective evaluation was carried out to determine the influence of app icon border form and interface background color saturation on user visual experience and search performance. The statistical results showed that during the human-computer interaction process, compared with the circle and rounded square border forms, the square icons had the longest search completion time and first fixation time and the lowest subjective evaluation score. In addition, rounded square icons had the best user visual experience. Compared with high-saturation background color, less time was required to complete search tasks in the low-saturation background color condition. Higher perceptual usability and aesthetic scores were also obtained under this condition compared to the high-saturation background color condition. Hence, this study confirmed that the two factors of icon border form and interface background color saturation had the basic properties of the interaction effect. The combination of rounded square icons and low-saturation background color can be beneficial to improve user search performance and visual experience. The findings can help in the design of human-computer interaction interfaces.

Static Model of Athlete's Upper Limb Posture Rehabilitation Training Indexes.

With the gradual expansion of the development of sports, the level of sports has been rapidly improved. Athletes have to carry out high-intensity and systemic technical movements in training and competition. Some sports have the greatest burden on the shoulder joint. From the observation and investigation of the injured parts of athletes, it is found that the shoulder joint is the most common sports injury, which is the most typical sports injury. Based on the problem of insufficient strength and endurance reserve after rehabilitation of shoulder external rotator injury, it will cause muscle tension and poor extensibility. To prove the improvement effect of functional training and posture index calibration on the poor posture of the shoulder, considering the measurement of global passive torque, this paper uses a limited set of joint angles and corresponding passive torque data in the upper arm lifting trajectory to train the neural network and uses the trained network to predict the passive torque in other upper arm trajectories. The kinematics model of the shoulder joint is established, and the human-computer interaction experiment is designed on the platform of the gesture index manipulator. The passive and active torque components of the shoulder joint in the human-computer interaction process are calculated by measuring the man-machine interaction force of the subjects in the motion state, which is used as the basis for evaluating the active motion intention of the subjects. Surface electromyography (SEMG) was used to calibrate and verify the attitude index of shoulder active torque. The method proposed in this paper is helpful to achieve more efficient on-demand assisted rehabilitation training exercises, which is of great significance to improve the level of rehabilitation training.

HCI – Human Computer Interaction Yesterday… Today… Tomorrow…

The theory of human-computer interaction is a new scientific direction today. Human-Computer Interaction (HCI) is a field of research and practice that developed in the early 1980s. Human-computer interaction (HCI) is an accompanying multidisciplinary field (e.g., engineering, psychology, ergonomics, design) that studies the theory, design, implementation, ways and methods of evolution of how people use computer devices and how they interact with them. Many researchers work to develop human-computer interaction. They represent a wide range of sciences: design, psychology, journalism ... Human computer interaction (HCI) specialists are in great demand in technology companies, which in turn influences the development of human computer interaction (HCI) training courses at universities. The paper discusses the basic theoretical and technological principles of HCI (Human Computer Interaction) human-computer interaction, based on the research papers of well-known scientists (Mandel Theo; Alan Dix; Janet Finlay; Russel Beale; John M. Carroll; Gregory Abowd; John Brooke; Jonathan Grudin; Emily LR Harriosn; Deborah Gail Tatar; Phoebe Sengers; Brad A. Myers; Christofer D. Wickens; John Lee; Yili D. Liu; Sallie Gordon-becker; Donald Norman; Elizabeth Churchill; Anne Bowser; Jennifer Preece; Gerard Jounghyum Kim; Hinze-Hoare Vita; Jakob Nielsen; Wilfred J. Hansen etc.) working in this field. The paper reviews the historical aspects of the development of human-computer interaction is; Its essence as a science is defined in the manuscript. The focus is on important aspects of human-computer interaction. Namely, the goals of human-computer interaction; Structure; Principles; Models; Systems; The principles of human-computer interface design are outlined, the importance of interface functionality and usability in the human-computer interaction process is shown; The concepts of interaction and interface are compared. Human-computer interaction approaches with current and future technologies are presented.

Emotional State Analysis Model of Humanoid Robot in Human-Computer Interaction Process

The traditional humanoid robot dialogue system is generally based on template construction, which can make a good response in the set dialogue domain but cannot generate a good response to the content outside the domain. The rules of the dialogue system rely on manual design and lack of emotion detection of the interactive objects. In view of the shortcomings of traditional methods, this study designed an emotion analysis model based on deep neural network to detect the emotion of interactive objects and built an open-domain dialogue system of humanoid robot. In affective state analysis language processing, language coding, feature analysis, and Word2vec research are carried out. Then, an emotional state analysis model is constructed to train the emotional state of a humanoid robot, and the training results are summarized.

Prediction of Human-Computer Interaction Intention Based on Eye Movement and Electroencephalograph Characteristics.

In order to solve the problem of unsmooth and inefficient human-computer interaction process in the information age, a method for human-computer interaction intention prediction based on electroencephalograph (EEG) signals and eye movement signals is proposed. This approach is different from previous methods where researchers predict using data from human-computer interaction and a single physiological signal. This method uses the eye movements and EEG signals that clearly characterized the interaction intention as the prediction basis. In addition, this approach is not only tested with multiple human-computer interaction intentions, but also takes into account the operator in different cognitive states. The experimental results show that this method has some advantages over the methods proposed by other researchers. In Experiment 1, using the eye movement signal fixation point abscissa Position X (PX), fixation point ordinate Position Y (PY), and saccade amplitude (SA) to judge the interaction intention, the accuracy reached 92%, In experiment 2, only relying on the pupil diameter, pupil size (PS) and fixed time, fixed time (FD) of eye movement signals can not achieve higher accuracy of the operator’s cognitive state, so EEG signals are added. The cognitive state was identified separately by combining the screened EEG parameters Rα/β with the eye movement signal pupil diameter and fixation time, with an accuracy of 91.67%. The experimental combination of eye movement and EEG signal features can be used to predict the operator’s interaction intention and cognitive state.

The Impact of Human and Computer Interaction on eLearning Quality

Over the last few years, particularly during the COVID-19 pandemic, the stage of education has clearly shifted from face-to-face instruction to e-learning. Human-Computer Interaction (HCI) paved the way for developing alternative and advanced methods for implementing the concept of e-learning by identifying what aids educational applications and platforms in visualizing content and thus providing students and their education with easy access in their homes during quarantine and providing them with easy access to educational materials throughout time. Students' impressions of the impact of the human-computer interaction process on the quality of e-learning at Tafila Technical University are discussed in this study. Within one year and four months of the COVID-19 quarantine, students got sufficient experience to apply e-learning. Students responded to a questionnaire that contained closed-ended questions. The responses were analyzed to ascertain the influence of the continual human-computer interaction process. The study's primary objective was to determine the beneficial effect of this contact on the quality of e-learning. Despite their academic and academic performance being greatly impacted by the COVID-19 epidemic, students' positive attitudes toward e-learning have been extremely enthusiastic.

Decision support for process operators: Task loading in the days of big data

Modern chemical processes rely on distributed control systems to make the repetitive and routine adjustments to maintain steady operation. Operators are still required to “supervise the (system) supervisor” and intervene when variables exceed pre-programmed parameters to avert major incidents. Research in human-computer interaction and advanced process control has often focused on data-driven methods for fault detection as distinct from operator effectiveness. In this paper, we explore the application of a novel data-driven fault-detection technique to enhance operator decision support. During a simulated abnormal event, three users attempted to diagnose the root cause of a process upset using a traditional or standard interface, then with the addition of causal maps, in a A-B-A single-subject design. The causal maps were derived using a hierarchical method that could be applied to a wide range of chemical processes as an online, adaptive augmentation for abnormal situation management. Using a think-aloud technique, the three participants developed high quality insights into the process without negatively impacting the overall task load. These preliminary findings challenge prevailing wisdom in process control interface design, which often focuses on de-cluttering displays at the cost of information resolution.

Research on Visual Awareness Interface Design Based on Human-Computer Interaction Task

By using the form and specific process of human-computer interaction to comprehensively understand and judge whether users can accurately and quickly recognize products, it has a positive effect on the current visual and perceptual interface design work. On the basis of understanding the achievements of current related scientific research projects, this paper analyzes how to construct and design a new visual and perceptual interface based on human-computer interaction tasks by studying the main factors that affect users’ cognition of visual information interface.

Computer Vision for Human-Computer Interaction Using Noninvasive Technology

Computer vision is a significant component of human-computer interaction (HCI) processes in interactive control systems. In general, the interaction between humans and computers relies on the flexibility of the interactive visualization system. Electromyography (EMG) is a bioelectric signal used in HCI that can be captured noninvasively by placing electrodes on the human hand. Due to the impact of complex background, accurate recognition and analysis of human motion in real-time multitarget scenarios are considered challenging in HCI. Further, EMG signals of human hand motions are exceedingly nonlinear, and it is important to utilize a dynamic approach to address the noise problem in EMG signals. Hence, in this paper, the Optimized Noninvasive Human-Computer Interaction (ONIHCI) model has been proposed to predict human motion recognition. Average Intrinsic Mode Function (AIMF) has been used to reduce the noise factor in EMG signals. Furthermore, this paper introduces spatial thermographic imaging to overcome the conventional sensor problem, such as gesture recognition and human target identification in multitarget scenarios. The human motion behavior in spatial thermographic images is examined by target trajectory, and body movement kinematics is employed to classify human targets and objects. The experimental findings demonstrate that the proposed method reduces noise by 7.2% and improves accuracy by 97.2% in human motion recognition and human target identification.