Human Machine Interface Control Research Articles

Cognitive task analysis-driven intelligent steering wheel interaction design

With the increasing levels of intelligence and automation, the relationship between humans and vehicles has evolved from a utilitarian perspective to a partnership. Among the crucial factors for enhancing user experiences are the analysis of driving tasks, the construction of user needs models, and the design of intelligent interfaces. Based on this background, this paper proposes a cognitive task analysis model using intelligent steering wheel information interaction design as the vehicle. The model aims to extract key design elements to assist designers in making design decisions, thereby improving the human-machine cooperation performance of intelligent automobiles and enhancing user perceptual experiences. Firstly, within the context of human-machine cooperation systems, a cognitive task analysis method integrating the SRK model is proposed. By analyzing the behavioral decision characteristics between the vehicle and the user, a framework for the human-machine interface (HMI) logic of the steering wheel and a dynamic layout prototype are established. Secondly, the design of the steering wheel’s HMI interaction is based on an analysis of users’ affective needs and rational physiological characteristics. This paper integrates the analysis of users’ affective needs to identify design elements that align with a high level of user satisfaction. Lastly, the design methodology model is applied to a navigation scenario, resulting in the creation of a steering wheel HMI prototype within a human-machine cooperation system. The prototype is then subjected to a combined subjective and objective experimental analysis, thereby validating the superiority of the steering wheel HMI’s detection indicators over those of the central control HMI and establishing the design pattern for the steering wheel HMI.

Human Machine Interface System for Filling and Capping Process

In this research, bottle filling and capping processes are performed simultaneously in the pharmaceutical factory by using the PC based human machine interface (HMI) control system. In the traditional method of bottle filling process, the bottles are placed onto a conveyor and only one bottle is filled at a time. So, this method is time consuming and expensive as wide working place and many human labors is required. In this research, the proposed design can solve this problem without any complex control by using the HMI automation control system. The monitoring and running conditions in the packaging system are shown on the monitor of computer by using the composed of Visual Basic.Net and Mikro C. It also includes the volume of medical powder is limited by changing the size of filling funnel which is set up of shutter at the top and bottom of funnel. The entire system is more flexible and time saving. A prototype is implemented by using the DC motor, sensing devices; limit switches, peripheral interface controller (PIC) device and serial port communication. These control systems is very flexible, cost effective, space efficient and reduce complexity. PC based HMI control is used for monitoring the process.

Binary Controller Based on the Electrical Activity Related to Head Yaw Rotation

A human machine interface (HMI) is presented to switch on/off lights according to the head left/right yaw rotation. The HMI consists of a cap, which can acquire the brain’s electrical activity (i.e., an electroencephalogram, EEG) sampled at 500 Hz on 8 channels with electrodes that are positioned according to the standard 10–20 system. In addition, the HMI includes a controller based on an input–output function that can compute the head position (defined as left, right, and forward position with respect to yaw angle) considering short intervals (10 samples) of the signals coming from three electrodes positioned in O1, O2, and Cz. An artificial neural network (ANN) training based on a Levenberg–Marquardt backpropagation algorithm was used to identify the input–output function. The HMI controller was tested on 22 participants. The proposed classifier achieved an average accuracy of 88% with the best value of 96.85%. After calibration for each specific subject, the HMI was used as a binary controller to verify its ability to switch on/off lamps according to head turning movement. The correct prediction of the head movements was greater than 75% in 90% of the participants when performing the test with open eyes. If the subjects carried out the experiments with closed eyes, the prediction accuracy reached 75% of correctness in 11 participants out of 22. One participant controlled the light system in both experiments, open and closed eyes, with 100% success. The control results achieved in this work can be considered as an important milestone towards humanoid neck systems.

A Human-Machine Interface Based on an EOG and a Gyroscope for Humanoid Robot Control and Its Application to Home Services

The human-machine interface (HMI) has been studied for robot teleoperation with the aim of empowering people who experience motor disabilities to increase their interaction with the physical environment. The challenge of an HMI for robot control is to rapidly, accurately, and sufficiently produce control commands. In this paper, an asynchronous HMI based on an electrooculogram (EOG) and a gyroscope is proposed using two self-paced and endogenous features, double blink and head rotation. By designing the multilevel graphical user interface (GUI), the user can rotate his head to move the cursor of the GUI and create a double blink to trigger the button in the interface. The proposed HMI is able to supply sufficient commands at the same time with high accuracy (ACC) and low response time (RT). In the trigger task of sixteen healthy subjects, the target was clicked from 20 options with ACC of 99.2% and RT 2.34 s. Furthermore, a continuous strategy that uses motion start and motion stop commands to create a certain robot motion is proposed to control a humanoid robot based on the HMI. It avoids the situation that combines some commands to achieve one motion or converts the certain motion to a command directly. In the home service experiment, all subjects operated a humanoid robot changing the state of a switch, grasping a key, and putting it into a box. The time ratio between HMI control and manual control was 1.22, and the number of commands ratio was 1.18. The results demonstrated that the continuous strategy and proposed HMI can improve performance in humanoid robot control.

Biosignal-Based Human-Machine Interfaces for Assistance and Rehabilitation: A Survey.

As a definition, Human–Machine Interface (HMI) enables a person to interact with a device. Starting from elementary equipment, the recent development of novel techniques and unobtrusive devices for biosignals monitoring paved the way for a new class of HMIs, which take such biosignals as inputs to control various applications. The current survey aims to review the large literature of the last two decades regarding biosignal-based HMIs for assistance and rehabilitation to outline state-of-the-art and identify emerging technologies and potential future research trends. PubMed and other databases were surveyed by using specific keywords. The found studies were further screened in three levels (title, abstract, full-text), and eventually, 144 journal papers and 37 conference papers were included. Four macrocategories were considered to classify the different biosignals used for HMI control: biopotential, muscle mechanical motion, body motion, and their combinations (hybrid systems). The HMIs were also classified according to their target application by considering six categories: prosthetic control, robotic control, virtual reality control, gesture recognition, communication, and smart environment control. An ever-growing number of publications has been observed over the last years. Most of the studies (about 67%) pertain to the assistive field, while 20% relate to rehabilitation and 13% to assistance and rehabilitation. A moderate increase can be observed in studies focusing on robotic control, prosthetic control, and gesture recognition in the last decade. In contrast, studies on the other targets experienced only a small increase. Biopotentials are no longer the leading control signals, and the use of muscle mechanical motion signals has experienced a considerable rise, especially in prosthetic control. Hybrid technologies are promising, as they could lead to higher performances. However, they also increase HMIs’ complexity, so their usefulness should be carefully evaluated for the specific application.

Intuitive real-time control strategy for high-density myoelectric hand prosthesis using deep and transfer learning

Myoelectric hand prostheses offer a way for upper-limb amputees to recover gesture and prehensile abilities to ease rehabilitation and daily life activities. However, studies with prosthesis users found that a lack of intuitiveness and ease-of-use in the human-machine control interface are among the main driving factors in the low user acceptance of these devices. This paper proposes a highly intuitive, responsive and reliable real-time myoelectric hand prosthesis control strategy with an emphasis on the demonstration and report of real-time evaluation metrics. The presented solution leverages surface high-density electromyography (HD-EMG) and a convolutional neural network (CNN) to adapt itself to each unique user and his/her specific voluntary muscle contraction patterns. Furthermore, a transfer learning approach is presented to drastically reduce the training time and allow for easy installation and calibration processes. The CNN-based gesture recognition system was evaluated in real-time with a group of 12 able-bodied users. A real-time test for 6 classes/grip modes resulted in mean and median positive predictive values (PPV) of 93.43% and 100%, respectively. Each gesture state is instantly accessible from any other state, with no mode switching required for increased responsiveness and natural seamless control. The system is able to output a correct prediction within less than 116 ms latency. 100% PPV has been attained in many trials and is realistically achievable consistently with user practice and/or employing a thresholded majority vote inference. Using transfer learning, these results are achievable after a sensor installation, data recording and network training/fine-tuning routine taking less than 10 min to complete, a reduction of 89.4% in the setup time of the traditional, non-transfer learning approach.

Virtual commissioning of automated manufacturing systems — Quality-handling station case study

The aim of the contribution is to create an event-controlled simulation of an automated production system using selected simulation software according to analyses and creation PLC control program compatible with human-machine control interface (HMI) visualization. The programmer can test the control program without the need for additional hardware on a virtual model of the machine. This method is called as a Virtual Commissioning method. The question is whether these new digital technologies are beneficial in the development process of new machines. The theoretical part of the contribution deals with the evaluation of the current state of knowledge in the field of virtual commissioning systems from the perspective of the Industry 4.0 concept and the use of principles taken from the technology of digital twins for analysing the possibilities of productive support of simulation, programming and visualization of control. For the Virtual Commissioning of the Quality-handling station from a flexible production system. It was chosen Tecnomatix Process simulating software for the developing of the Digital Twin of the station that is suitable for the connection of the logical behaviour inputs and outputs of the Digital Twin to the PLCSIM emulator. The main goal is to create a functional PLC program, so that in the second phase the process of virtual commissioning can be tested by the Software-the-Loop method. The preparation of the simulation model consists in a sample of the control station model of the production system, the process of creating kinematic bonds, behaviour model, creating control signals in the Tecnomatix Process Simulate program. Programming procedure and acquaintance with control means in the Simatic Manager program, selection of communication interface and creation of control visualization. In the final work, validation is performed and evaluation supports the objectives and the results confirmed the benefits of the Virtual Commissioning tool.

Установка для экспериментальных исследований многофазных электромеханических систем

The subject of this study is an installation for experimental research designed to study the characteristics and control algorithms of multiphase motors with the number of working phases from 3 to 8, connected by a star, a triangle, or in another way, allowing phase currents to flow, creating a rotating electromagnetic field. The installation consists of two separate independent units: a controller, or a human-machine control interface, and a power inverter module (converter). The controller is connected to the converter by a two-wire half-duplex interface (RS485) via the Modbus RTU communication protocol. The installation also includes synchronous motors with the number of phases 3, 5, 7.Using the developed installation for experimental research, it is possible to carry out experimental studies of multiphase motors when implementing various control algorithms for a converter that implements pulse-width vector modulation. The time required to implement control algorithms is minimal. According to the results of the experiments, it is possible to carry out a comparative analysis of multiphase motors in terms of energy efficiency, in terms of vibration of electromagnetic origin, in dynamic parameters. An experimental assessment of the load of the converter keys is possible.The created installation is an effective tool for checking the reliability of the results of theoretical studies of electromechanical systems based on multiphase motors.

Robust Cascade Temperature Control for a HSNWT

Applies programmable logic controller (PLC) to configure control hardware system, addresses the temperature control for a high speed nitrogen gas wind tunnel (HSNWT) testing, in order to obtain variable high temperature control with variable high speed gas flow, a coordinating factor of the gas flow is proposed between the tunnel’s heating and water cooling systems, moreover, considering the temperature modeling nonlinearities, uncertainties and disturbances, establish robust cascade fuzzy PID and expert predictive control strategies for the inner-loop water cooling subsystem and the outer-loop heating system, respectively, which effectively overcomes the influences of large inertia and transport time delay on the temperature responses. Furthermore, designs human-machine integrated user control interface (HMI), achieves fast and accurately control for user operating. The designed system are simulated and tested in the application, which results demonstrate that the system runs stable and reliable, has strongly robustness when the temperature changes with different loading heat modes, and has excellent capability of variable high speed nitrogen gas flow.

Prediction of Optimal Facial Electromyographic Sensor Configurations for Human-Machine Interface Control.

Surface electromyography (sEMG) is a promising computer access method for individuals with motor impairments. However, optimal sensor placement is a tedious task requiring trial-and-error by an expert, particularly when recording from facial musculature likely to be spared in individuals with neurological impairments. We sought to reduce the sEMG sensor configuration complexity by using quantitative signal features extracted from a short calibration task to predict human-machine interface (HMI) performance. A cursor control system allowed individuals to activate specific sEMG-targeted muscles to control an onscreen cursor and navigate a target selection task. The task was repeated for a range of sensor configurations to elicit a range of signal qualities. Signal features were extracted from the calibration of each configuration and examined via a principle component factor analysis in order to predict the HMI performance during subsequent tasks. Feature components most influenced by the energy and the complexity of the EMG signal and muscle activity between the sensors were significantly predictive of the HMI performance. However, configuration order had a greater effect on performance than the configurations, suggesting that non-experts can place sEMG sensors in the vicinity of usable muscle sites for computer access and healthy individuals will learn to efficiently control the HMI system.

PULSED ELECTRIC FIELD FOR PASTEURIZATION OF FRESH SUGARCANE JUICE

Pasteurization methods using pulsed electric fields (PEFs) have proved efficient at non-thermal inactivation of microorganisms and pathogens. The methods usually use a series of high intensity, narrow band electric fields that pass through target mediums where they induce irreversible damage to cellular membranes leading to the death of microorganisms by internal content loss breakage of cell homeostasis and other effects. This research proposed a simple, lab-scale PEF pasteurization equipment that uses common output electric voltages (220V and 110V, which provide electric power of 5.5 and 2.75 kV/m, respectively) with a simple designed Human-Machine Interface control. At a voltage of 110V, the equipment was tested with non-sterilized Czapek medium which showed more than 1.42-log reduction in the microbial density at 30s of the treatment and temperature below 60 o C. A test on suspended E. coli in modified Czapek medium also indicated an inactivation of most of bacterial cells after 40s of the treatment. We then applied the equipment to pasteurize fresh sugarcane juice and, as a result, the content of native microorganisms in sugarcane juice was reduced 1.2-log cycle, compared to that of original juice before the treatment.

Context-dependent multimodal communication in human-robot collaboration

In the dynamic environment of human-robot collaboration, a key for boosting the efficiency of human workers is supporting them with context-dependent work instructions, delivered via communication modalities that suit the actual context. Workers, in turn, should be supported in controlling the robot or other components of the production system by using the most convenient modality, thus lifting the limitations of traditional interfaces as push buttons installed at fixed locations. We introduce a workflow for context-dependent multimodal communication in a collaborative work environment and present its implementation in a Human-Machine Interface Controller system.

Toward Intuitive Prosthetic Control: Solving Common Issues Using Force Myography, Surface Electromyography, and Pattern Recognition in a Pilot Case Study

Despite the appearance of advanced multi-degrees of freedom (DoF) robotic hands during the past decade, prosthetic control lacks a powerful interface to facilitate all its functionalities in a manner that is acceptable for a majority of users [1]. In this article, we explore the feasibility of using a sensing technique called force myography (FMG) as an alternative or synergist to the traditional surface electromyography (sEMG) technique as a human-machine interface (HMI) for the control of a multi-DoF prosthetic hand, bebionic 3 by Ottobock, Austin, Texas. In this article, we present a prosthetic prototype developed for the Cybathlon 2016, a championship for racing pilots with disabilities using assistive robotic devices. The design of the prototype is discussed and the effect of two factors on its control is analyzed. These factors are 1) the impact of a multisensory approach and 2) the placement of FMG sensor strips within the prosthetic inner socket. Analysis is performed by comparing resulting pattern recognition accuracies. Results show that the use of both sensing modalities (FMG and EMG) together produced the highest pattern recognition accuracy (81.1%) for ten classes of motion (four wrist movements and six grip patterns). We demonstrated that FMG has the potential to be an HMI for control of upper-limb-powered prostheses. FMG also illustrates the potential for intuitive control through the use of pattern recognition. A multisensory approach could assist in increasing robustness of the HMI for prosthetic control.

A fuzzy approach towards inductive transfer and human–machine interface control design

Traditional machines do not adapt to their operators, instead they implicitly demand human adaptation. Human adaptive mechatronics (HAM) is the research topic that covers the design of devices and controllers for assisting the human. HAM devices are capable to measure and estimate the operator’s skill/dexterity, while a real-time assist-controller enhances machine adaptation, improving the overall human–machine performance. Nowadays, the demand for such devices has particular potential in many activities, which involve manual operations, such as in assistive technology. The main contribution of this work is the proposal of a fuzzy clustering methodology to the development of a real-time inductive transfer embedded controller, used for improving the operator’s proficiency, under a human-in-the-loop environment relying on visual feedback information. Other contribution is the proposal of a condition for inductive transfer between human operators, based on correlation analysis. The operator behaviour is modelled and enhanced from a human–machine interface fuzzy classifier and assisting scheme, which uses real-time data and additional information collected from an expert user. Experimental tests were performed by different participants under a driving simulator, for evaluation of the proposed methodology. The fuzzy clustering approach confirmed to significantly improve the transfer learning and the driving skills of the human operators.

Using noise to shape motor learning.

Many theories argue that we choose to make the specific movements that minimize motor noise. Here, by changing the relationship between movements and noise, we show that people actively learn to make movements that minimize noise. This not only provides direct evidence for the theories of noise minimization but presents a way to use noise to teach specific movements to improve rehabilitative therapies and human-machine interface control.

Development of a Human Machine Interface for Control of Robotic Wheelchair and Smart Environment

In this work, we address the problem of integrating a robotic wheelchair into a smart environment. This approach allows people with disabilities to control home appliances of the environment using a Human Computer Interface (HCI) based on different biological signals. The home appliances includes TV, radio, lights/lamp and fan. Three control paradigms using surface Electromyography (sEMG), Electrooculography (EOG) and Electroencephalography (EEG) signals were used. These signals are captured through a biosignal acquisition system. Three sub-paradigms for sEMG/EOG analyzes were defined: moving eyes horizontally (left/right), raising brow and prolonged clench. On the other hand, the navigation of the wheelchair is executed through an Steady-State Visually Evoked Potentials (SSVEP)-BCI. Each stage of our proposed system showed a good performance for most subjects. Therefore, volunteers were recruited to participate of the study and were distributed in two groups (subjects for home appliances and subjects for SSVEP-BCI). The average accuracy for prolonged clench approach was of 95%, the raising brow was 85% and moving eyes achieved 93%. Multivariate Synchronization Index (MSI) was used for feature extraction from EEG signals. The flickering frequencies were 8.0 Hz (top), 11.0 Hz (right), 13.0 Hz (bottom) and 15.0 Hz (left). Results from this approach showed that classification varies in the range of 45-77% among subjects using window length of 1 s.

A simple ERP method for quantitative analysis of cognitive workload in myoelectric prosthesis control and human-machine interaction.

Common goals in the development of human-machine interface (HMI) technology are to reduce cognitive workload and increase function. However, objective and quantitative outcome measures assessing cognitive workload have not been standardized for HMI research. The present study examines the efficacy of a simple event-related potential (ERP) measure of cortical effort during myoelectric control of a virtual limb for use as an outcome tool. Participants trained and tested on two methods of control, direct control (DC) and pattern recognition control (PRC), while electroencephalographic (EEG) activity was recorded. Eighteen healthy participants with intact limbs were tested using DC and PRC under three conditions: passive viewing, easy, and hard. Novel auditory probes were presented at random intervals during testing, and significant task-difficulty effects were observed in the P200, P300, and a late positive potential (LPP), supporting the efficacy of ERPs as a cognitive workload measure in HMI tasks. LPP amplitude distinguished DC from PRC in the hard condition with higher amplitude in PRC, consistent with lower cognitive workload in PRC relative to DC for complex movements. Participants completed trials faster in the easy condition using DC relative to PRC, but completed trials more slowly using DC relative to PRC in the hard condition. The results provide promising support for ERPs as an outcome measure for cognitive workload in HMI research such as prosthetics, exoskeletons, and other assistive devices, and can be used to evaluate and guide new technologies for more intuitive HMI control.

Discrete and continuous auditory feedback based on pitch and spatial lateralization for human-machine-interface control

The purpose of this study was to investigate auditory-motor learning via discrete and continuous auditory feedback using pitch and spatial lateralization. Sixteen subjects used facial surface electromyography (sEMG) to control a human-machine-interface (HMI). The output of the HMI was a lateralized harmonic tone. The fundamental frequency and lateralization (left-right ear) changed with the sum and the difference of the bilateral muscle signals. For eight participants, these changes were continuous, whereas the other eight participants received discrete feedback, in which the frequency of the tone was one of nine possible semitones (from midi note #64 to #75) and the lateralization was either left, center or right. Participants trained over three days. A mixed-models analysis of variance showed a significant effect of learning over sessions and a trend for increased performance for the group utilizing discrete feedback. Overall, information transfer rates using this purely auditory feedback averaged 38.5 bit/min by day 3, which is similar to results from similar systems utilizing visual feedback. These results show that with minimal training, auditory feedback can provide usable HMI control.

Combined Auditory and Vibrotactile Feedback for Human–Machine-Interface Control

The purpose of this study was to determine the effect of the addition of binary vibrotactile stimulation to continuous auditory feedback (vowel synthesis) for human-machine interface (HMI) control. Sixteen healthy participants controlled facial surface electromyography to achieve 2-D targets (vowels). Eight participants used only real-time auditory feedback to locate targets whereas the other eight participants were additionally alerted to having achieved targets with confirmatory vibrotactile stimulation at the index finger. All participants trained using their assigned feedback modality (auditory alone or combined auditory and vibrotactile) over three sessions on three days and completed a fourth session on the third day using novel targets to assess generalization. Analyses of variance performed on the 1) percentage of targets reached and 2) percentage of trial time at the target revealed a main effect for feedback modality: participants using combined auditory and vibrotactile feedback performed significantly better than those using auditory feedback alone. No effect was found for session or the interaction of feedback modality and session, indicating a successful generalization to novel targets but lack of improvement over training sessions. Future research is necessary to determine the cognitive cost associated with combined auditory and vibrotactile feedback during HMI control.

ARM BASED HUMAN MACHINE INTERFACE OF PLASTIC EXTRUSION BLOW MOLDING SYSTEM

This paper designs monitoring and controlling parameters of Plastic Extrusion Blow molding system. This system takes ARM processor as a core and takes CAN bus as communication system by analyzing the human machine interface system. In this Hardware design consist of LPC2138 with ARM7 microprocessor having large memory capacity and CAN communication as Human machine interface control device. In software design Embedded C is transplanted to the ARM core to achieve real time multi task switching. Secondly, GUI is transplanted to display graphics & text. Write the HMI application program task-unit. Finally LPC 2138 communicates to human machine interface system with CAN bus and HMI of the Plastic Extrusion blow molding system is completed. Index Terms: Human Machine Interface (HMI), LPC2138, CAN bus (controller Area Network) GUI (Graphical User Interface)