Co-Adaptation in Human–Machine Interaction: What We Learned and Where To Go Next

Comprehensive view of occupational safety research: Prioritizing topics in robotics and autonomous machines. Barriers to safety research: Logistical, intellectual property, timeline, and funding challenges. Importance of surveillance or tracking system: Documenting fatalities, injuries, and near misses/good catches. Priority safety research needs: human-machine interaction, adoption of automation in the work setting, and surveillance/tracking. Collaboration with technology developers: Overcoming barriers and exploring emerging technologies and potential safety implications. In 2022, the SAfety for Emerging Robotics and Autonomous AGriculture (SAFER AG) Workshop was held to discuss and understand emerging challenges related to safety, occupational safety research needs, workforce implications, and other issues associated with robotics and autonomous machines in agriculture. This paper presents the major findings from the occupational safety research track of the workshop. This track identified existing hurdles to conducting occupational safety research including logistical barriers, intellectual property concerns, long timelines, and lack of funding. Considerations for developing a tracking or surveillance system for adverse events as well as exposure related to these technologies were also discussed, emphasizing the need for a comprehensive system. Finally, the priority occupational safety research needs identified during the session were related to human and non-human machine interaction, adoption of automation in the work setting, and event tracking/surveillance. To overcome barriers to research, collaboration between occupational safety researchers and technology developers is crucial. Enhancements to existing surveillance systems can facilitate better understanding of captured events. Additionally, prioritizing research on worker risk from robotics and autonomous machines in agriculture is essential. The integration of robotics and autonomous machines in agriculture has revolutionized the industry but requires evidence-based safety research, outreach, and education to ensure worker safety and health.

This study is based on the human-machine interaction that is the way of communication between automatic systems and people. The HMI process allows humans to control the machine through initiatives behaviours and natural. Those factors also lead to building the established communication process with the technology and digital system as well. From this study, it has been seen that H2M devices include a sensor for measuring the comments signal such as voice, touch, and gestures for constructing an updated system to control and a measurement system to measure the electrophysiological signals. This study has also focused on the Sensor HMI that can help to detect the temperature and strain pressure. It can provide the opportunity to monitor and mimic human senses and it also can improve machine control by graphene, carbon nanotubes, hydrogel composites, and Ag nanowires. These kinds of elements give the provision to build a stretchable sensor for developing the structure's stretchability. Accordingly, it can be said that the mechanical sensor process assists to apply the attachment of the enabling strain sensors in the finger joints for measuring the movements of the figure and using the pressure sensor for improving the analysis of grasping activities. This study has also shed light on the Actuators of HMI that helps to deliver all kinds of human output and also have the ability to mimic the sense of people or individual for the desired outcome of human and machine interaction. It can help to stimulate the senses of humans mechanically. In this context, it can be said that wearable gloves use the benefits of soft actuators for rehabilitation. It helps to support the force of grasping. On the other hand, the mechanical vibration process helps to stimulate the human sense that helps to provide certain information as well. Intelligence-aided devices usage is one of the other critical parts of this study and from this portion, it has been seen that the advancement of actuators and sensors in machine intelligence help to get higher detectability in soft sensors, coping with chronic problems, and decoupling unwanted stimuli in sensor process. This factor helps human hands to provide the highest degree of freedom in the body for performing a range of tasks with the development of detecting hand motions. This study has chosen a secondary process to get the data for the study and it also helps to make a proper conclusion for the study. At last, it can be said that HMI is a blessing for the world population and future generations. Keywords : Ag nanowires, carbon nanotubes, human-machine interaction (HMI), and hydrogel compos PDMS.

Recently consumer electronics products for the human device or machine interaction in smart healthcare systems have been widespread due to progress in health monitoring hardware and remote diagnostic services. Affect recognition through thermal facial signatures is significant in real-time Human-machine Interaction (HMI) studies. The distribution of facial skin temperature displays explicit characteristics related to affect arousal. During human interacts with a machine or computer, it is challenging to detect face and track the facial regions of interest (ROI) in a thermal video because of head motion artifacts. Our proposed embedded portable HMI product integrated with the required hardware and deep learning architecture is used to recognize the human affect in real-time. The transfer learning approach (Faster R-CNN) and the Multiple Instance Learning (MIL) algorithm are applied to thermal video for thermal face and ROIs detection and tracking. The multivariate time series (MTS) features generated from specific ROIs' calculated mean intensity variations. This study proposes a k-Nearest Neighbor (k-NN) algorithm-based graph neural network (GNN) architecture that utilizes MTS features to recognize affects in the facial thermal video. The effectiveness of the proposed algorithm for the In-house and NVIE thermal data-set gives better accuracy of 76.05% and 71.91% compared with state-of-the-art approaches.

The popularization of intelligent machines such as service robot and industrial robot will make human–machine interaction, an essential work mode. This requires employees to adapt to the new work content through learning. However, the research involved human–machine interaction that how influences the employee’s learning is still rarely. This paper was to reveal the relationship between human–machine interaction and employee’s learning from the perspective of job characteristics and competence perception of employees. We sent questionnaire to 500 employees from 100 artificial intelligence companies in China and received 319 valid and complete responses. Then, we adopted a hierarchical regression for the test. Empirical results show that human–machine interaction has a U-shaped curvilinear relationship with employee learning, and employee’s vitality mediates the curvilinear relationship. In addition, job characteristics (skill variety and job autonomy) moderate the U-shaped curvilinear relationship between human–machine interaction and employee’s vitality, especially the results of moderating effects varying with employee’s competence perception. Exploring the mechanism of the effect of human–machine interaction on employee’s learning enriches the socially embedded model. Moreover, it provides managerial implications how to enhance individual adaptability with the introduction of AI into firms. However, our research focuses more on the impact of human–machine interaction on employees at the initial stage of AI development, and the level of machine intelligence in various industries will reach a high degree of autonomy in the future. The future research can explore the impact of human–machine interaction on individual’s behavior at different stages, and the results may vary depending on the technologies mastered by different individuals. The study has theoretical and practical significance to human–machine interaction literature by underscoring the important of individual’s behavior among individuals with different skills.

Human-machine interactive systems show increasing demand for analysing fast moving objects in high-frame-rate videos. Robust foreground detection, which is able to reduce large amount of redundant background data from high-frame-rate video, becomes the essence to achieve ultra-high-speed human-machine interactions. This paper proposes a local spatial propagation based background model generation, a local linear illumination correction based background model update, and a regional central coordinates and edge keypoints constrained foreground region reselection. The three proposals make up a robust and hardware-friendly foreground detection method. Experimental results prove that the proposed hardware-friendly algorithm achieves high accuracy and robustness on various kinds of challenging cases. Meanwhile, the hardware implementation utilizes little hardware resources and achieves realtime processing of high-frame-rate (784 frame/second) video with the delay less than 1 ms/frame in image processing core. In addition, a practical system is implemented by combing a PC, a high-speed camera and a field programmable gate array (FPGA) for realworld applications. This work will significatively promote the development and application of high-speed human machine interaction. A demo of the proposed vision system working at 784 FPS is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://wcms.waseda.jp/em/5f84f75136a6</uri> . Note to Practitioners—This paper was motivated by the problem of high-frame-rate video contains large amount of redundant background pixels which makes ultra-high-speed human-machine interactions inaccessible. Existing approaches are mainly focused on designing complex background models, but processing speed, which is the most important issue for ultra-high-speed human-machine interactions, has received relatively little attention. This paper suggests a robust and hardware-friendly foreground detection algorithm which has been implemented as a hardware system by using an FPGA, a high-frame-rate camera, and a PC. We show that the hardware implementation utilizes less hardware resources and achieves real-time processing speed of 784 FPS with the delay less than 1 ms/frame in the image processing core. This work is a pioneering attempt of ultra-high-speed foreground detection, which will significatively speed up the wide applications of ultra-high-speed human machine interactions.

Human Machine Interaction (HMI) is the study of the interaction between humans and machines, and how these interactions can be optimized to create more intuitive, efficient, and effective interfaces. HMI encompasses a wide range of technologies and disciplines, including computer science, engineering, psychology, and design. At its core, HMI is concerned with creating interfaces that allow humans to interact with machines in a way that is natural and intuitive, using modalities such as touch screens, voice recognition, augmented reality, virtual reality, AI/ML. The goal of HMI is to make machines more accessible and usable for humans, while also improving the performance and functionality of the machines themselves. HMI is an area of rapid innovation and development, with new technologies and techniques being developed all the time. Some of the key trends in HMI include the use of natural language processing, wearable and IoT, emotion recognition, brain-computer interfaces, and explainable AI. The applications of HMI are wideranging, encompassing everything from consumer electronics such as smart phones and tablets to industrial automation systems and medical devices. By creating more intuitive and effective interfaces between humans and machines, HMI has the potential to revolutionize the way we interact with technology and the world around us

As Human machine interaction (HMI) is becoming increasingly complex, the need to assess the efficiency of modes of human-machine communication are increasingly becoming relevant. Several methods to analyze the cognitive activity of users for HMI exist like verbal protocols, cognitive task analysis and physiological measures (EEG, MEG & fMRI) etc. In this paper, power spectrum analysis of EEG in parietal and occipital lobes of the human brain has been used for comparative study of efficiency of visual and auditory tasks in HMI. For the same task, both the visual and auditory instructions were given and the observed efficiency correlated with the EEG power spectrum. The results showed that visual channel lead to less brain activity and hence was faster means of machine to human communication when the subject was attending to the task. Further, this paper argues for EEG power spectrum as an objective measure of the human cognitive load caused by the machine, environment and task in the HMI setup.

Wearable patch with direction-aware sensitivity of in-plane force for self-powered and single communication channel based human-machine interaction

Negative driving emotions constitute a significant factor compromising road safety. Current intelligent vehicle human machine interaction (HMI) systems predominantly focus on functional implementation, lacking the capability to perceive and adapt to the driver’s psychological state. To address this issue, this study investigates the intrinsic relationship between driving emotions and HMI through multimodal experiments. Experiment One reveals the distribution patterns of drivers’ visual attentional scope under different emotional states. Experiment Two establishes a color preference model for HMI interfaces corresponding to specific emotions. Experiment Three quantitatively analyzes the impact of emotional variations on the perceptual efficiency of auditory warnings. Based on the experimental data, an interaction design principle matching “Emotion-Scene-Modality” is formulated, guiding the design of a data-driven, emotion-adaptive HMI prototype system. This system can perceive the driver’s emotional state in real time via multimodal sensors and dynamically adjust interface color themes, information layout, warning sound effects, and voice interaction style according to predefined interaction strategies. Usability testing demonstrates that, compared to traditional static HMI, this affective adaptive system effectively mitigates the driver’s negative emotional load and provides alerts that are more perceptible and less likely to cause irritation during critical moments. Consequently, it offers a significant theoretical foundation and practical reference for constructing a safer and more comfortable next-generation intelligent vehicle cockpit interaction paradigm.

In the mobile industrial human machine interaction (HMI) based on wireless sensor networks, the engineer has to manually select the target machine from a long list which may lead to wrong connection and waste of time. We observe that the engineer should face to the machine during the interaction to ensure that the machine works accurately, and this characteristic makes the proximity estimation algorithm suitable to simplify the data connection. However, due to the densely deployed machines and the frequent HMI in the industrial plant, the existing algorithms can not provide sufficient accuracy with limited latency. In this paper, we implement a mobile industrial HMI testbed to evaluate the characteristics of wireless communication in the face-to-machine HMI, and then propose the face-to-machine proximity estimation (FaceME) algorithm based on the analytical results. The experimental results prove that FaceME can provide guaranteed estimation accuracy and low time complexity that satisfies the requirements of the face-to-machine HMI.

Since the concept of Industry 5.0 was proposed, the emphasis on human–machine​ interaction (HMI) in industrial scenarios has continued to increase. HMI is part of the factory’s development towards Industry 5.0, mainly because HMI can help realise the human-centric vision. At the same time, to achieve the sustainable and resilient goals proposed by Industry 5.0, green, smart, and more advanced technologies are also considered important driving factors for factories to achieve Industry 5.0. Human-centric smart manufacturing (HCSM) factories that integrate HMI with advanced technologies are expected to become the paradigm of future manufacturing. Therefore, it is necessary to discuss technologies and research directions that may promote the implementation of HCSM in the future. In a smart factory, HMI signals will go through the process of being collected by sensors, processed, transmitted to the data analysis centre and output to complete the interaction. Based on this process, we divide HMI into four parts: sensor and hardware, data processing, transmission mechanism, and interaction and collaboration. Through a systematic literature review process, this article evaluates and summarises the current research and technologies in the HMI field and categorises them into four parts of the HMI process. Since the current usage scenarios of some technologies are relatively limited, the introduction focuses on the possible applications and problems they face. Finally, the opportunities and challenges of HMI for Industry 5.0 and HCSM are revealed and discussed.

Similarities and differences of emotions in human–machine and human–human interactions: what kind of emotions are relevant for future companion systems?

A pandemic situation like COVID-19 calls for safe transactions that include: free from touch or physical movement or using common platforms or using common devices or doing from home. The safer the transaction, the more customer satisfaction. On one hand, banks have become the lifeblood of every human today because any transaction payment from grocery to gold/house demands interaction of human–machine rather than manual payments. On the other hand, the services of banks should ensure ease, convenience, comfort, and security in every transaction to the customer so that the transactions are fast and delivered rapidly. The level of customer satisfaction extrapolates the future platform of banks. The present study highlights the importance of human–machine interaction in pandemic situations. The research study is based on primary and secondary data to examine the relationship between banking services involving human–machine interactions to the level of customer satisfaction. SPSS Software is used to analyze the data. The results of the study highlight positivity thus concluding that the twenty-first century is demanding a hundred percent of transactions based on human–machine interaction and also defines the future status of banks’ existence.KeywordsHuman–machine interactione-banking servicesSPSS software

The paradigm of human-centered multimedia computing (HCMC) has emerged recently as a result of the increasing emphasis on integrating the concept of human-centeredness in various aspects of multimedia computing. While many theories have been proposed to advance this paradigm, it is our belief that a complete understanding of the issues surrounding HCMC requires capturing a complementary (yet enriching) perspective through inspirations drawn from studying human disabilities and deficits. In this paper, we present the need for understanding human deficiencies in sensory, neural, and cognitive sensing/actuations which could reveal innate components of human interaction that benefits researchers, designers and developers of new multimedia solutions. We illustrate how technologies that were started with assistive and rehabilitative goals have broader impacts to the general population. More importantly, this opens up new research issues that would otherwise not have been seen when the focus is only on the 'able' population. The study and understanding of the disabilities and deficits leads to a better understanding of human requirements in any human machine interaction which is important in advancing the vision and core principles of HCMC.

Time-consuming purchases and operation failures could often occur in the utilization of Vending Machines (VMs) due to the violations between human purchase behavior and design interfaces. An unhindered, clear and concise selfservice shopping mood needs to be discovered under the study of human purchase behavior and characteristics of Human- Machine Interaction (HMI) process in order to solve these problems. In this paper, each step of human's habitual purchase behaviors on VMs were analyzed so that the features of operations were extracted. Thenkey points of psychological load and mistake operation were located by investigating the operation on the screen of VMs and being combined with the found features of purchase behavior so that the HMI process can be optimized. Lastly, experiments were conducted so that a conclusion can be drawn that the improved screen operation procedures did reduce the time of purchase and times of operation failure. This study and the validating framework could not only provide a model to solve the HMI of VM problems, but also lay a foundation for future research on the new way of machine-based sales system.