Human-machine Interaction Process Research Articles

Intelligent Cockpits for Connected Vehicles: Taxonomy, Architecture, Interaction Technologies, and Future Directions.

Highly integrated information sharing among people, vehicles, roads, and cloud systems, along with the rapid development of autonomous driving technologies, has spurred the evolution of automobiles from simple "transportation tools" to interconnected "intelligent systems". The intelligent cockpit is a comprehensive application space for various new technologies in intelligent vehicles, encompassing the domains of driving control, riding comfort, and infotainment. It provides drivers and passengers with safety, comfort, and pleasant driving experiences, serving as the gateway for traditional automobile manufacturing to upgrade towards an intelligent automotive industry ecosystem. This is the optimal convergence point for the intelligence, connectivity, electrification, and sharing of automobiles. Currently, the form, functions, and interaction methods of the intelligent cockpit are gradually changing, transitioning from the traditional "human adapts to the vehicle" viewpoint to the "vehicle adapts to human", and evolving towards a future of natural interactive services where "humans and vehicles mutually adapt". This article reviews the definitions, intelligence levels, functional domains, and technical frameworks of intelligent automotive cockpits. Additionally, combining the core mechanisms of human-machine interactions in intelligent cockpits, this article proposes an intelligent-cockpit human-machine interaction process and summarizes the current state of key technologies in intelligent-cockpit human-machine interactions. Lastly, this article analyzes the current challenges faced in the field of intelligent cockpits and forecasts future trends in intelligent cockpit technologies.

In-situ enhanced anchor-free deep CNN framework for a high-speed human-machine interaction

Human-Robot Interaction (HRI) constitutes a demanding research field that integrates artificial intelligence, informatics, robotics, engineering, and human-machine interaction. In the present era, there is an increased focus on natural user interfaces, with particular attention to gestural modalities. This leads to the coordination of modern robotic systems, where real-time integration and interpretation of manual gestures play a vital role in facilitating effective manipulation. Hand gesture recognition holds a crucial role in the field of machine vision, perhaps greatly exacerbated with varying lighting conditions and backgrounds. In this paper, first an Enhanced Anchor-free Network (EAF-Net) is proposed to perform hand gesture recognition in real-time. The EAF-Net is deep single-stage Convolutional Neural Network (CNN) based architecture. The Custom Precise Prediction Function (CPPF) is utilized for a continuous recognition of a specific gesture from a video feed. Then, the 6-axis collaborative Robot is manipulated using the predicted gesture. The EAF-Net model utilizes the Enriched Hourglass as a backbone feature extraction network. The proposed EAF-Net model undergoes training and assessment using the MITI HD-II dataset. The performance of the model is also analyzed for the standard benchmark datasets (NUSHP-II and Senz-3D). The evaluation of the EAF-Net model’s performance spans a range of IoU values from 0.5 to 0.95. The EAF-Net model led to higher precision values (AP0.5) as 99.22%, recall values (AR0.5) as 98.20%, and F1-Score0.5 as 98.64% on the MITI HD-II dataset. The EAF-Net model consumed a prediction time of 259 ms on NVIDIA Jetson Nano Processor and 14 ms on NVIDIA Titan X GPU. It aided in implementing a portable high-speed 6-axis Robot control framework to pick and place operation in 3D space. The robot would be able to imitate human hand activities such as hand rotation, hand tilting, picking and arranging items, and so on in the near future. Further reduction in prediction time of the model on the portable processors (NVIDIA Jetson Nano, Raspberry-Pi) could enhance the speed of the Human Machine Interaction process.

Wearable patch with direction-aware sensitivity of in-plane force for self-powered and single communication channel based human-machine interaction

The merging of flexible technologies with human machine interaction (HMI) is now optimizing the way people communicates with electrical terminals. Compared with “press” and “strain”, “sliding” is a directional operation which requires the interface to identify the directions of the applied force for accurate interaction. Previous efforts that explore the “directional force” in HMI system are mainly based on sensor array, which brings concern of complex electrode design and multiple communication channels to prevent cross-talk. In view of this, we developed a self-powered and wearable HMI interface that can distinguish the axial directions of in-plane force based on Faraday’s law of induction. The interface consists of well-orientated magnetized micropillars, and a conductive coil that collects and transmits the electrical signals during the interaction process. When in-plane sliding force was applied, distinguishable signals were generated to reflect the different axial directions (+X/−X/+Y/−Y) according to the polarity and number of voltage peaks. With this unique behavior, the HMI process can be completed with two electrodes and one communication channel in an interference-free manner. Through the systematic optimization, the intrinsic oscillation from the micropillars results in obviously enhanced signals for a high accuracy and reliability towards real application. The MMPs-based interface was successfully established for HMI platforms such as intelligent robot control, and Morse code communication, etc. Owing to the robustness, humid resistance, accuracy and reliability, we expect that the interface design can inspire the development of flexible and wearable devices in HMI especially for scenarios that require a high command capacity.

A Framework for Service-Oriented Digital Twin Systems for Discrete Workshops and Its Practical Case Study

To address issues in discrete manufacturing workshops, such as the difficulty for management personnel to coordinate workshop production and the challenge of visualizing and supervising a massive amount of temporary data, this paper proposes a service-oriented digital-twin-system framework for discrete workshops using the industrial IoT platform as the system-service platform to solve the problems of the opaque monitoring of operators in discrete workshops, the low interactivity of 2D monitoring systems, and the difficulty of the visual monitoring of workshop data. Firstly, the current situation of intelligent manufacturing workshop-monitoring demand in the context of new-generation information technology is analyzed, and a six-dimensional digital-twin-workshop-monitoring architecture is proposed, whereby a discrete workshop monitoring system based on the digital twin is constructed with IoT as the service platform. We will conduct research on the construction of virtual workshops for the system development process, twin data collection based on edge computing gateways, and dynamic monitoring of the production process. Finally, through the application of this system framework in a movable-arm-production workshop, the more intelligent human–machine interaction process of browsing and controlling workshop information, such as the equipment layout and production processes in the virtual workshop, has been realized. This includes data acquisition based on edge-computing gateways, dynamic real-time monitoring of the production process, etc., which provides a reference for realizing the visual monitoring of the discrete workshop.

Effectiveness of Human Machine Interaction (HMI) for Development of Communication and Interaction between a Human and a Machine to Control Machines

This study is based on the human-machine interaction that is the way of communication between automatic systems and people. The HMI process allows humans to control the machine through initiatives behaviours and natural. Those factors also lead to building the established communication process with the technology and digital system as well. From this study, it has been seen that H2M devices include a sensor for measuring the comments signal such as voice, touch, and gestures for constructing an updated system to control and a measurement system to measure the electrophysiological signals. This study has also focused on the Sensor HMI that can help to detect the temperature and strain pressure. It can provide the opportunity to monitor and mimic human senses and it also can improve machine control by graphene, carbon nanotubes, hydrogel composites, and Ag nanowires. These kinds of elements give the provision to build a stretchable sensor for developing the structure's stretchability. Accordingly, it can be said that the mechanical sensor process assists to apply the attachment of the enabling strain sensors in the finger joints for measuring the movements of the figure and using the pressure sensor for improving the analysis of grasping activities. This study has also shed light on the Actuators of HMI that helps to deliver all kinds of human output and also have the ability to mimic the sense of people or individual for the desired outcome of human and machine interaction. It can help to stimulate the senses of humans mechanically. In this context, it can be said that wearable gloves use the benefits of soft actuators for rehabilitation. It helps to support the force of grasping. On the other hand, the mechanical vibration process helps to stimulate the human sense that helps to provide certain information as well. Intelligence-aided devices usage is one of the other critical parts of this study and from this portion, it has been seen that the advancement of actuators and sensors in machine intelligence help to get higher detectability in soft sensors, coping with chronic problems, and decoupling unwanted stimuli in sensor process. This factor helps human hands to provide the highest degree of freedom in the body for performing a range of tasks with the development of detecting hand motions. This study has chosen a secondary process to get the data for the study and it also helps to make a proper conclusion for the study. At last, it can be said that HMI is a blessing for the world population and future generations. Keywords : Ag nanowires, carbon nanotubes, human-machine interaction (HMI), and hydrogel compos PDMS.

Human Reliability Evaluation Based on Objective and Subjective Comprehensive Method Used for Ergonomic Interface Design

In the process of machining, the operator’s incorrect operation using the control human-computer interaction interface may lead to a disastrous effect on the whole production chain such as task failure and system failure. The eight types of using errors about human-computer interaction interface system (misperception, memory lapse, carelessness, improper decision-making, improper operation, improper interface layout design, improper icon and text display design, and improper prompt feedback) are proposed based on the cognition from users. According to the eight types of using error, the human reliability and safety evaluation method is proposed. Considering the variability objective and uncertainty subjective factors in the human-machine interaction process, the human reliability evaluation method is proposed based on objective and subjective comprehensive weighting under ergonomic principles. Firstly, based on the investigation of relevant machine tool interface design, the comprehensive evaluation index system for man-machine reliability of complex interface design is established. Secondly, an objective and subjective comprehensive weight assignment method as the composition of Theilʼs entropy weight method combined with TOPSIS (Technique for Order Preference by Similarity to an Ideal Solution) and improved AHP (analytic hierarchy process) method combined with fuzzy mathematics is proposed. It can not only effectively avoid the impact of excessive subjective factors on empowerment but also fully excavate the real information in the survey data and improve the objectivity and accuracy of the weight data of evaluation indicators at all levels. Thirdly, taking the interface design of CNC machine tool equipment for example, the best human-computer interaction interface design schemes are chosen according to the human reliability evaluation method based on objective and subjective comprehensive method calculated by Entropy-TOPSIS-AHP. Finally, based on eye movement experiments, the use efficiency and accuracy of the human-computer interaction interface design scheme chosen by the assessment method are better than the other design schemes or original one to verify the assessment method validity.

Crowdsourcing label quality: a theoretical analysis

Crowdsourcing has been an effective and efficient paradigm for providing labels for large-scale unlabeled data. In the past few years, many methods have been developed for inferring labels from the crowd, but few theoretical analyses have been presented to support this popular human-machine interaction process. In this paper, we theoretically study the quality of labels inferred from crowd workers by majority voting and provide an analysis of label quality that shows that the label error rate decreases exponentially with the number of workers selected for each task. We also study the problem of eliminating low-quality workers from the crowd, and provide a conservative condition for eliminating low-quality workers without eliminating any non-low-quality worker with high probability. We also provide an aggressive condition for eliminating all low-quality workers with high probability.

Theorizing change in artificial intelligence: inductivising philosophy from economic cognition processes

Economic value additions to knowledge and demand provide practical, embedded and extensible meaning to philosophizing cognitive systems. Evaluation of a cognitive system is an empirical matter. Thinking of science in terms of distributed cognition (interactionism) enlarges the domain of cognition. Anything that actually contributes to the specific quality of output of a cognitive system is part of the system in time and/or space. Cognitive science studies behaviour and knowledge structures of experts and categorized structures based on underlying structures. Knowledge representation through understanding of `epistemic cultures' is an evolutionary stage. But cognition goes beyond knowledge representation. Notwithstanding the importance of epistemology of phenomena, the practicability cum philosophical aspects of machine learning needs to be seen in dynamic behaviour in socio-economic-technical value additions if human machine interaction processes that are context specific are incorporated into strong artificial intelligent systems. Cognitive Science is also studied from both computational and biological angles. Evolution of interactive forms of reasoning through understanding of meta-language of computations or biological learning processes is possible. But the limitation of historical cultures predefines the role of interactive processes in user-networks beyond technology networks. Despite this limitation, inclusive development notions of a heterogeneous national society such as India or Europe can be tested and incorporated.

Construction of Prediction Models of Cognitive Load in Human-Machine Interaction Process

Complex operations required in modern human-machine systems often increase cognitive load on human operators.Under excessive cognitive load,human operators may exhibit delayed information processing or even make some mistakes,probably inducing various operating accidents.Therefore,an assessment and prediction of the general situation of the cognitive load will be needed to help design the system with proper cognitive load on the operators.Also,this will be of great significance to improve operators' working environment,job satisfaction and operational security.The present study attempted to establish effective prediction models to evaluate cognitive load changes in human-machine interaction processes.A dual-task experiment with simulated web-searching as the primary task and mental arithmetic as the secondary task was conducted.Twenty six volunteers,aged 20 to 27 years,participated in this study.Each participant performed sixteen phases of the task.The order of task phases was randomized across participants.Finally,three types of indices were obtained,including task-performance measures (response time and accuracy of the primary task),subjective evaluations (mental efforts and perceived task difficulty) and eye movement measures (gaze durations and the number of gazes).The effects of task duration on cognitive load were examined using these different indices.The BP and Elman neural network models were developed to predict the changing tendency of cognitive load in the different phases of the task.The results showed that (1) Task duration had a significant effect on the response time and accuracy of the primary task,gaze durations,and the number of gazes respectively.However,it had no significant effect on either mental efforts or perceived task difficulty.(2) The BP and Elman neural network models could predict the changes in the relevant assessment indices of cognitive load in the different phases of the task.Furthermore,a synthetic analysis could be made in combination with the synthetic assessment model of cognitive load in the different phases of the task.The current findings suggest that neural network models can be used to predict the cognitive load changes on human operators in complex human-machine systems efficiently.The BP and Elman neural network models developed here may be the good approaches to the dynamic self-adaptive allocation of tasks in modern complex systems,while the models developed in the current study may not be generally appropriate for different human-machine interactions.In a sense,the importance of the present findings is that they provide an effective approach as to how to predict the cognitive load changes in human-machine interaction processes.

Synthetic Assessment of Cognitive Load in Human-Machine Interaction Process

Cognitive load (CL) often becomes too great in modern human-machine systems,which affects the performance and reliability of system operators. Therefore an effective and reliable approach to measure CL is crucial. Currently there are three major measures of CL:these are subjective evaluation,performance measures and physiological assessment. None of these,however,can comprehensively reflect the state of CL under different task conditions. Thus,this study attempted to establish an appropriate multi-dimensional assessment technique to evaluate CL synthetically in the human-machine interaction process. A dual-task experiment with simulated web-searching as the primary task and mental arithmetic as the secondary task was conducted. The primary task had three levels of complexity,high,medium,and low. Thirty-two subjects were selected to perform the dual -task with each complexity level of the primary task. The order of performance of the three complexity levels was randomized across subjects. Four types of indices were obtained,including primary-task measures,secondary-task measures,subjective evaluations and eye movement measures. The synthetic assessment models were constructed via factors analysis,BP artificial neural network and self-organizing feature map,respectively. The results showed that:(1) The six indices,which are mental effort,perceived task difficulty,duration of fixation,number of fixations,response time and the percentage of correct responses in the primary task,were sensitive to the change in cognitive load. (2) In dual-task situations,the multidimensional synthetic assessment of cognitive load generated more valid results than those based on the single assessment index. (3) According to the indices of relative,absolute and average errors,the two ANN models (BP neural network and self-organizing feature map) showed a more accurate measurement of cognitive load than the traditional technique of factors analysis.