Human-machine Systems Research Articles

Human–Machine Cooperative Driving and Real-Time Algorithm Implementation Based on Double-Loop Control System

The double loop structure has been implemented to verify the feasibility of human-machine cooperative control in real environment in this paper, which can fully reflect the performance of the controller and realization of the sharing steering control between human driver and automatic control system. The outer loop adopts fuzzy proportional-integral-derivative(PID) controller to deal with different working conditions, on which the complexity of the algorithm is reduced and the real-time performance of the controller is improved. The inner loop adopts the active disturbance rejection controller (ADRC), which can effectively suppress the interference such as drivers’ faulty operations etc. and improve the tracking accuracy and system performance. The stability for extended state observe(ESO) and ADRC is described in theory to ensure the feasibility of the controller in the real world environment. The experiment results show that the human-machine cooperative control system based on the double-loop controller can effectively reduce the lateral error of the vehicle and the steering torque of the driver in the process. And the control performance of the vehicle is improved.

Skin in the Game: The Transformational Potential of Decentralized Autonomous Organizations

Decentralized autonomous organizations (DAOs)—collectively owned human-machine systems deployed on a blockchain that self-govern through smart contracts and the voluntary contributions of autonomous community members—exhibit the potential to facilitate collective action in managing digital commons. Yet the promise of decentralization and collective action is difficult to sustain. To this end, this paper critically examines the transformational potential of DAOs in the case of decentralized finance. Using a polycentric governance lens, we contribute to the literature on technology-enabled forms of organizing with a model explaining the transformational potential of DAOs to facilitate collective action in digital commons. Our study highlights that (1) DAOs are a new form of organizing enabled by blockchain technology in which individuals are free to pursue their objectives within a general system of rules enforced by smart contracts, (2) collective action for managing digital commons can be sustained through a set of three mechanisms—sustained participation, collective direction, and scaled organizing, and (3) DAOs tend to strike a balance between centralized and fully decentralized or community-based governance by implementing a polycentric governance system involving a combination of human and machine agency that creates skin in the game.

Optimization of Torque-Control Model for Quasi-Direct-Drive Knee Exoskeleton Robots Based on Regression Forecasting.

The choice of torque curve in lower-limb enhanced exoskeleton robots is a key problem in the control of lower-limb exoskeleton robots. As a human-machine coupled system, mapping from sensor data to joint torque is complex and non-linear, making it difficult to accurately model using mathematical tools. In this research study, the knee torque data of an exoskeleton robot climbing up stairs were obtained using an optical motion-capture system and three-dimensional force-measuring tables, and the inertial measurement unit (IMU) data of the lower limbs of the exoskeleton robot were simultaneously collected. Nonlinear approximations can be learned using machine learning methods. In this research study, a multivariate network model combining CNN and LSTM was used for nonlinear regression forecasting, and a knee joint torque-control model was obtained. Due to delays in mechanical transmission, communication, and the bottom controller, the actual torque curve will lag behind the theoretical curve. In order to compensate for these delays, different time shifts of the torque curve were carried out in the model-training stage to produce different control models. The above model was applied to a lightweight knee exoskeleton robot. The performance of the exoskeleton robot was evaluated using surface electromyography (sEMG) experiments, and the effects of different time-shifting parameters on the performance were compared. During testing, the sEMG activity of the rectus femoris (RF) decreased by 20.87%, while the sEMG activity of the vastus medialis (VM) increased by 17.45%. The experimental results verify the effectiveness of this control model in assisting knee joints in climbing up stairs.

Estimating and approaching the maximum information rate of noninvasive visual brain-computer interface

An essential priority of visual brain-computer interfaces (BCIs) is to enhance the information transfer rate (ITR) to achieve high-speed communication. Despite notable progress, noninvasive visual BCIs have encountered a plateau in ITRs, leaving it uncertain whether higher ITRs are achievable. In this study, we used information theory to study the characteristics and capacity of the visual-evoked channel, which leads us to investigate whether and how we can decode higher information rates in a visual BCI system. Using information theory, we estimate the upper and lower bounds of the information rate with the white noise (WN) stimulus. Consequently, we found out that the information rate is determined by the signal-to-noise ratio (SNR) in the frequency domain, which reflects the spectrum resources of the channel. Based on this discovery, we propose a broadband WN BCI by implementing stimuli on a broader frequency band than the steady-state visual evoked potentials (SSVEPs)-based BCI. Through validation, the broadband BCI outperforms the SSVEP BCI by an impressive 7 bps, setting a record of 50 bps. The integration of information theory and the decoding analysis presented in this study offers valuable insights applicable to general sensory-evoked BCIs, providing a potential direction of next-generation human-machine interaction systems.

Design of Human-Machine Integration System to Meet Diverse Interactive Tasks

A flexible production model in which humans and robots collaborate has become an urgent need for industrial production. Therefore, research on human-machine integration systems has received more and more attention in recent years. Human-computer interaction is widely used in industrial production and entertainment life, such as industrial control, VR games, etc. In this article, we designed and implemented a human-machine fusion system that meets different interactive tasks. Human movement estimation and robot interactive control are the key technologies. First, we propose a human posture solution method based on information fusion of multiple cameras and inertial measurement units. This method constructs an optimization problem, fuses the 2D joint detection information under multiple cameras and the measurement information of the worn inertial measurement unit, and optimizes the estimation of human kinematic posture, which improves the problems of incomplete posture information and sensitivity to noise under a single sensor. Therefore, this method improves the accuracy of pose estimation. Secondly, combining the kinematic characteristics of the robot and the characteristics of human-computer interaction, we designed a robot control strategy based on target point tracking and model predictive control, so that the robot can adapt to dynamic environments and different interaction needs by adjusting control parameters. At the same time, our design ensures the safety of robots and operators. Finally, we conducted human-computer interaction experiments such as action following, item delivery, and active obstacle avoidance. The experimental results show the effectiveness and reliability of the designed robot interaction system in a human-machine fusion environment.

Syllable-rate-adjusted-modulation (SRAM) predicts clear and conversational speech intelligibility.

Objectively predicting speech intelligibility is important in both telecommunication and human-machine interaction systems. The classic method relies on signal-to-noise ratios (SNR) to successfully predict speech intelligibility. One exception is clear speech, in which a talker intentionally articulates as if speaking to someone who has hearing loss or is from a different language background. As a result, at the same SNR, clear speech produces higher intelligibility than conversational speech. Despite numerous efforts, no objective metric can successfully predict the clear speech benefit at the sentence level. We proposed a Syllable-Rate-Adjusted-Modulation (SRAM) index to predict the intelligibility of clear and conversational speech. The SRAM used as short as 1 s speech and estimated its modulation power above the syllable rate. We compared SRAM with three reference metrics: envelope-regression-based speech transmission index (ER-STI), hearing-aid speech perception index version 2 (HASPI-v2) and short-time objective intelligibility (STOI), and five automatic speech recognition systems: Amazon Transcribe, Microsoft Azure Speech-To-Text, Google Speech-To-Text, wav2vec2 and Whisper. SRAM outperformed the three reference metrics (ER-STI, HASPI-v2 and STOI) and the five automatic speech recognition systems. Additionally, we demonstrated the important role of syllable rate in predicting speech intelligibility by comparing SRAM with the total modulation power (TMP) that was not adjusted by the syllable rate. SRAM can potentially help understand the characteristics of clear speech, screen speech materials with high intelligibility, and convert conversational speech into clear speech.

Machine Learning-Enabled Environmentally Adaptable Skin-Electronic Sensor for Human Gesture Recognition.

Stretchable sensors have been widely investigated and developed for the purpose of human motion detection, touch sensors, and healthcare monitoring, typically converting mechanical/structural deformation into electrical signals. The viscoelastic strain of stretchable materials often results in nonlinear stress-strain characteristics over a broad range of strains, consequently making the stretchable sensors at the body joints less accurate in predicting and recognizing human gestures. Accurate recognition of human gestures can be further deteriorated by environmental changes such as temperature and humidity. Here, we demonstrated an environment-adaptable high stress-strain linearity (up to ε = 150%) and high-durability (>100,000 cycles) stretchable sensor conformally laminated onto the body joints for human gesture recognition. The serpentine configuration of our ionic liquid-based stretchable film enabled us to construct broad data sets of mechanical strain and temperature changes for machine learning-based gesture recognition. Signal recognition and training of distinct strains and environmental stimuli using a machine learning-based algorithm analysis successfully measured and predicted the joint motion in a temperature-changing environment with an accuracy of 92.86% (R-squared). Therefore, we believe that our serpentine-shaped ion gel-based stretchable sensor harmonized with machine-learning analysis will be a significant achievement toward environmentally adaptive and multianalyte sensing applications. Our proposed machine learning-enabled multisensor system may enable the development of future electronic devices such as wearable electronics, soft robotics, electronic skin, and human-machine interaction systems.

Improving Supply Chain Human-Machine Systems by the Analysis of Departmental-Level User Characteristics

In this study, the authors aimed to enhance supply chain efficiency at Company A, a key player in China's manufacturing sector, by focusing on user-centric system design. The approach involved analyzing departmental-level user characteristics to inform the development of an optimized human-machine interface. To achieve this, they conducted focus groups and card sorting exercises, identifying specific user needs across departments like planning, purchasing, receiving, warehousing, production, and quality control. The application of these insights into system design led to significant operational improvements. Notably, time savings were observed in departments such as receiving (74.44%) and quality control (41.86%). This tailored approach underscored the importance of understanding and integrating user characteristics in supply chain systems, leading to enhanced productivity and a competitive edge. The study demonstrates the impact of user-focused system design in improving both efficiency and user satisfaction in supply chain management.

Многокритериальная оценка антропогенного риска опасности электроустановок на основе имитационного моделирования

Risk management in a human-machine system is considered. The basic models and methods of decision-making, “man - electrical installation - environment” are presented. The method of risk analysis is considered. An expert software complex is proposed that implements the risk assessment of electrical installations. Keywords: HUMAN-MACHINE SYSTEM, RISK MANAGEMENT, ELECTRICAL INSTALLATION, MULTICRITERIA ANALYSIS, RISK ANALYSIS METHOD, “BOW TIE”, FIRE RISK OF ELECTRICAL INSTALLATIONS, SIMULATION MODEL

A systemic approach for stochastic reliability management in human–machine systems

In today’s complex engineered systems, comprising a multitude of interacting components, preserving system performance is of utmost importance. The challenge often lies in effectively prioritizing components with the highest potential to compromise system reliability, mainly when human interaction with technical artefacts is not negligible. This study proposes a systemic methodology for pragmatic reliability management within human–machine systems. The proposed approach combines a rule-based adaptation of the well-established Failure Mode, Effects, and Criticality Analysis (FMECA) with a probabilistic Fault Tree Analysis (FTA). Furthermore, the technical considerations are seamlessly integrated into a human-centric analysis, utilizing the Standardized Plant Analysis Risk – Human Reliability Analysis (SPAR-H). The proposed decision-support methodology is instantiated through Monte Carlo simulations to account for stochastic phenomena and uncertain operating conditions. The effectiveness and practicality of the proposed approach are elucidated through a case study involving a high-reliability system, specifically a high-mobility multi-wheeled vehicle. This study demonstrates the step-by-step application of the proposed approach and its implications in challenging operating scenarios, reaffirming its potential to enhance reliability management within human–machine systems.

Human-machine interactions based on hand gesture recognition using deep learning methods

Human interaction with computers and other machines is becoming an increasingly important and relevant topic in the modern world. Hand gesture recognition technology is an innovative approach to managing computers and electronic devices that allows users to interact with technology through gestures and hand movements. This article presents deep learning methods that allow you to efficiently process and classify hand gestures and hand gesture recognition technologies for interacting with computers. This paper discusses modern deep learning methods such as convolutional neural networks (CNN) and recurrent neural networks (RNN), which show excellent results in gesture recognition tasks. Next, the development and implementation of a human-machine interaction system based on hand gesture recognition is discussed. System architectures are described, as well as technical and practical aspects of their application. In conclusion, the article summarizes the research results and outlines the prospects for the development of hand gesture recognition technology to improve human-machine interaction. The advantages and limitations of the technology are analyzed, as well as possible areas of its application in the future.

A Path to Industry 5.0 Digital Twins for Human–Robot Collaboration by Bridging NEP+ and ROS

The integration of heterogeneous hardware and software components to construct human-centered systems for Industry 5.0, particularly human digital twins, presents considerable complexity. Our research addresses this challenge by pioneering a novel approach that harmonizes the techno-centered focus of the Robot Operating System (ROS) with the cross-platform advantages inherent in NEP+ (a human-centered development framework intended to assist users and developers with diverse backgrounds and resources in constructing interactive human–machine systems). We introduce the nep2ros ROS package, aiming to bridge these frameworks and foster a more interconnected and adaptable approach. This initiative can be used to facilitate diverse development scenarios beyond conventional robotics, underpinning a transformative shift in Industry 5.0 applications. Our assessment of NEP+ capabilities includes an evaluation of communication performance utilizing serialization formats like JavaScript Object Notation (JSON) and MessagePack. Additionally, we present a comparative analysis between the nep2ros package and existing solutions, illustrating its efficacy in linking the simulation environment (Unity) and ROS. Moreover, our research demonstrates NEP+’s applicability through an immersive human-in-the-loop collaborative assembly. These findings offer promising prospects for innovative integration possibilities across a broad spectrum of applications, transcending specific platforms or disciplines.

안전문화 구축을 위한 시뮬레이션과 메타버스와의 관계 연구

With the advent of the 4th industrial revolution due to the development of the communication industry, convergence between systems is actively taking place, with IoT, VR, Metaverse and AI at the center of it. Currently, simulators are used for education and simulation training in order to improve the safety culture or reduce human error in the main tasks of power facilities. However, detailed information on the phenomena occurring in power facilities during education and simulation training is not provided. Therefore, in this study, in order to improve the understanding of the relationship between driving operation and power facilities (human-machine system), we went one step further from the 3D educational video that has been used as an auxiliary material so far, and grafted virtual reality (VR) technology. This is a study on how to establish an integrated safety culture of electric power facility simulation training device (HMI), virtual reality (VR) and metaverse that can be directly experienced. And, as a research method to achieve the purpose of this study, a method combining OPC communication and Socket communication, which enables large-capacity signal processing, multi-connection, and two-way communication, was used. Because it can reduce human error, it responds to similar environments such as disasters, and furthermore, BCMS (Business Simulation Training in connection with the Continuity Management System) is also possible. In the future, it is expected that the concept to be conceived through this study will be able to proceed with additional research on the metaverse and safety culture construction related to the psychological therapy effect of ergonomics compared to the age of artificial intelligence.

Метод оптимизации антропогенных рисков опасности электроустановок

Currently, the creation of processes for the functioning of human-machine systems of the “Ch-E-S” type in terms of efficiency and reliability is the most popular area in the field of automation, control and decision-making. Intelligent decision support systems help decision makers analyze significant amounts of initial information, taking into account expert assessments of specialists, formulate many possible solution options and find the best one. This article discusses the problem associated with the development of an algorithm for creating many alternative options in the process of functioning of a human-machine system, allowing for its optimization. Keywords: OPTIMIZATION, ANTHROPOGENIC RISKS, HUMAN-MACHINE SYSTEM, EXPERT ASSESSMENT, DECISION MAKING MODEL, “BOW TIE” METHOD

Методы анализа безопасности электроустановок в человекомашинных системах

Increasing the intensification of agricultural production and Improving the quality of life of the rural population is a priority in the development of the agricultural sector of the country's economy. In this regard, the problem reducing man-made threats of accidents and injuries to people, fires, as well as increasing the reliability and safety of electrical installations, including electrical networks and technological electrical equipment is currently important and timely. This problem is especially acute as a result of the aging of the industry's fixed assets. The electric power park of the Russian agro-industrial complex mainly consists of exhausted electrical equipment, the wear of which exceeds a critical level (~ 70%): The state of technological safety of electrical installations today cannot be considered satisfactory. Every year, more than 4,500 people die in the public utility sector alone, about 30 thousand become disabled, while the countryside accounts for about 70% of the total number of electrical injuries. Fires caused by electric current as a result of short circuits and damage to the insulation of electrical wiring also pose a serious threat. The current situation poses a threat to the national security of the country, which necessitated the inclusion of the problem of technological safety in the list of critical technologies approved by the President of the Russian Federation in 2002. Keywords: ELECTRICAL INSTALLATION, RISK, HUMAN-MACHINE SYSTEM, SOFT COMPUTING, DAMAGE, SIMULATION MODELING, EXPERT SYSTEM

Имитационное моделирование и методы определения уровня пожарной безопасности электроустановок на объектах АПК

Currently, problems of health protection and ensuring anthropogenic human safety are becoming relevant. Their solution is impossible without an effective system for monitoring the environment and predicting its dynamics. The peculiarity of such a system is the spatial heterogeneity and temporal nonstationarity of its inherent patterns, which generate complexity and uncertainty of the human-machine system being studied. The work examines two interrelated tasks: 1. Generalization of the main provisions of simulation modeling as a methodology for studying complex human-machine systems under a priori uncertainty; 2. Methods for determining the level of fire safety electrical installations at agricultural facilities. Keywords: SIMULATION MODELING, FIRE SAFETY, HUMAN-MACHINE SYSTEM

Operator Role Classification in Human-Automation Interaction: A Systematic Review

Technical advancements, such as autonomous systems, prompt a reevaluation of operator roles within human-machine systems in manufacturing. This paper presents a systematic review of 370 records concerning applications in relevant application areas. It offers classifications of activities within human-machine systems, providing a nuanced exploration of operator roles and interaction dynamics. Through this analysis, three distinct perspectives emerge, focusing on agent dependency, information processing, and functional roles. The results highlight the importance of clear role descriptions and the current lack of standardized approaches and offering valuable insights for future research and practical applications.

Spatial-Frequency Characteristics of EEG Associated With the Mental Stress in Human-Machine Systems.

Accurate assessment of user mental stress in human-machine system plays a crucial role in ensuring task performance and system safety. However, the underlying neural mechanisms of stress in human-machine tasks and assessment methods based on physiological indicators remain fundamental challenges. In this paper, we employ a virtual unmanned aerial vehicle (UAV) control experiment to explore the reorganization of functional brain network patterns under stress conditions. The results indicate enhanced functional connectivity in the frontal theta band and central beta band, as well as reduced functional connectivity in the left parieto-occipital alpha band, which is associated with increased mental stress. Evaluation of network metrics reveals that decreased global efficiency in the theta and beta bands is linked to elevated stress levels. Subsequently, inspired by the frequency-specific patterns in the stress brain network, a cross-band graph convolutional network (CBGCN) model is constructed for mental stress brain state recognition. The proposed method captures the spatial-frequency topological relationships of cross-band brain networks through multiple branches, with the aim of integrating complex dynamic patterns hidden in the brain network and learning discriminative cognitive features. Experimental results demonstrate that the neuro-inspired CBGCN model improves classification performance and enhances model interpretability. The study suggests that the proposed approach provides a potentially viable solution for recognizing stress states in human-machine system by using EEG signals.

Neural network algorithm for predicting human speed based on computer vision and machine learning

The problem of increasing the accuracy of predicting human actions is an urgent task for various human-machine systems. The study examines the solution to the problem of predicting human speed using neural network algorithms, computer vision technologies, and machine learning. The formalization and software implementation of a neural network speed prediction algorithm are presented. To solve the problems of determining the current speed and predicting the upcoming positions of the human body depending on the dynamics of its movement, a comparison of various machine learning models was carried out. The RandomForestRegressor algorithm showed the best position prediction accuracy. The best determination of the current speed was demonstrated by dense multilayer neural networks. The experiment revealed that when predicting a person's position at an interval of 0.6 seconds, his speed is determined with an accuracy of more than 90%. The results obtained can be used to implement neural network algorithms for controlling human-machine systems.

Development of a stress-free algorithm for controlling active running platforms

The problem of increasing the accuracy of predicting human actions is an urgent task for various human-machine systems. The study examines the solution to the problem of predicting human speed using neural network algorithms, computer vision technologies, and machine learning. The formalization and software implementation of a neural network speed prediction algorithm are presented. To solve the problems of determining the current speed and predicting the upcoming positions of the human body depending on the dynamics of its movement, a comparison of various machine learning models was carried out. The RandomForestRegressor algorithm showed the best position prediction accuracy. The best determination of the current speed was demonstrated by dense multilayer neural networks. The experiment revealed that when predicting a person's position at an interval of 0.6 seconds, his speed is determined with an accuracy of more than 90%. The results obtained can be used to implement neural network algorithms for controlling human-machine systems.