Machine Interface System Research Articles

Self-powered and degradable humidity sensors based on silk nanofibers and its wearable and human–machine interaction applications

Humidity sensors have received increasing attention due to their broad potential application in wearable devices, health monitoring, intelligent control, and other fields. However, traditional humidity sensors are prone to produce various toxic substances to pollute the environment after discarding. Also, they are prone to damage during use, which shortens their service life. Therefore, the development of eco-friendly, degradable, and repairable humidity sensors is of great significance for reducing e-waste and protecting the environment. Here, we started with the selection of materials by using silk nanofiber (SNF) film as the substrate material, copper/aluminum as electrodes, and calcium chloride as the electrolyte, combined with the working mechanism of the primary battery, to construct a self-powered humidity sensor. The humidity sensor exhibits excellent humidity sensing performance and good linearity within the range of 30–90 % relative humidity (RH). A single humidity sensor can output 346 mV at 90 % RH. After being damaged, the humidity sensor can be repaired with the help of water. When the ruptured SNF film encounters water molecules, the water molecules can prompt the hydrogen bonds between the SNFs to rearrange and connect, connecting the broken area. Young’s modulus retention rate of the repaired SNF film is over 85 %. The repaired humidity sensor can also output 328 mV. Additionally, an alkaline solution can completely degrade the humidity sensor in less than 80 min. The proposed humidity sensors have been applied to wearable self-powered wristbands, wireless monitoring systems, and non-contact human–machine interaction systems. Hence, we provide comprehensive research from the material level (SNFs) to the functional device level (the self-powered humidity sensor), and then to the system level (wearable and human–machine interface system). The research will open a new way for the design of the self-powered humidity sensor and have great potential applications in wearable electronics and human–machine interface applications.

Modeling risk potential fields for mandatory lane changes in intelligent connected vehicle environment

In the environment of Intelligent Connected Vehicles (ICVs), the ICV system is capable of gathering, processing, and transmitting the majority of traffic information to each vehicle, thereby empowering drivers to make more informed and secure decisions. Nevertheless, ICVs are still incapable of completely avoiding traffic conflicts and accidents. Consequently, developing a risk assessment model for ICVs is crucial for their deployment. To investigate the dynamic patterns of lane-changing vehicle risks within ICV environments, this study developed a quantitative relationship model based on the Artificial Potential Field (APF) theory. The model correlates the Total Potential Field (TPF) of mandatory lane-changing (MLC) vehicle with the longitudinal/transverse gravitational potential field (LGPF/TGPF) and the longitudinal/transverse repulsion potential field (LRPF/TRPF). Subsequently, three sets of Human Machine Interface (HMI) systems were developed to delineate distinct ICV environments: baseline, warning group, and guidance group. The baseline furnishes fundamental data, the warning group supplements preceding vehicle icons and real-time headway information, whereas the guidance group additionally augments speed and voice guidance functionalities. Following this, a simulated driving experiment involving 43 participants was carried out to simulate MLC scenarios in highway merging sections. Using enumeration and gradient descent methods, the quantitative relationship between different types of risk fields was determined. The findings indicated that the ICV environment significantly impacts the mean values of LGPF, TGPF, LRPF (Leading Vehicle, LV), and TRPF, while insignificantly affecting the mean value of LRPF (Following Vehicle, FV). Nevertheless, a notable discrepancy in the mean value of LRPF (FV) was observed between the warning and guidance groups. Comparative results with lane positions demonstrate that the proposed TPF accurately depicts vehicle lane-changing behaviors. Comparative results with classical risk indicators indicate that TPF more accurately depicts the actual driving risks and variation trends faced by vehicles in different MLC processes. The research findings will be pivotal in improving real-time control and guidance strategies for vehicles in highway merging sections under ICV environments.

Closed-Loop Brain Machine Interface System for In-Vehicle Function Controls Using Head-Up Display and Deep Learning Algorithm

Closed-Loop Brain Machine Interface System for In-Vehicle Function Controls Using Head-Up Display and Deep Learning Algorithm

The Effect of Directional Tactile Memory of the Back of the User on Reaction Time and Accuracy

Tactile memory is the cognitive process of storing and recalling information that has been perceived through the sense of touch. Directional tactile memory involves the encoding and retrieval of sensory data associated with a tactile experience, allowing individuals to remember and recognize directional information encoded through the sense of touch. A new method for providing directional tactile feedback, at the back of the user, has been developed to investigate the efficacy of directional tactile memory, its decay over time, and its impact during a concurrent cognitive task. Two experiments were presented. In the first experiment, tactile memory deterioration, with a visual or a tactile cue, was tested with different action-cue latencies (10 s and 20 s). In the second experiment, we considered tactile memory deterioration when there was an increased cognitive load as the participants played Tetris. Forty volunteers participated in the two experiments using purpose-built tactile seats with nine motors controlled by an Arduino. The performance data (error and reaction times) were analyzed statistically, and a NASA task load index (NASA-TLX) questionnaire was administered to measure the subjective workload after each of the two experiments. The findings highlighted that the directional tactile memory of the back can guide individuals to the correct point on the screen and that it can be maintained for at least 20 s. There was no statistically significant difference in the number of errors or reaction time with a visual or tactile action cue. However, being involved in a concurrent cognitive task (playing Tetris) adversely affected the reaction time, the number of errors, and the directional tactile memory, which degraded as the time between the directional cue and the action cue increased. Participants perceived the performance while playing Tetris as significantly more mentally and perceptually demanding, requiring more mental and physical effort and being more frustrating. These trials revealed a new potential for a human–machine interface system, leveraging directional tactile memory, which might be utilized to increase the safety of autonomous vehicles.

Genetic Algorithm-Based Data Optimization for Efficient Transfer Learning in Convolutional Neural Networks: A Brain–Machine Interface Implementation

In brain–machine interface (BMI) systems, the performance of trained Convolutional Neural Networks (CNNs) is significantly influenced by the quality of the training data. Another issue is the training time of CNNs. This paper introduces a novel approach by combining transfer learning and a Genetic Algorithm (GA) to optimize the training data of CNNs. Transfer learning is implemented across different subjects, and the data chosen by GA aim to improve CNN performance. In addition, the GA-selected data shed light on the similarity in brain activity between subjects. Two datasets are used: (1) the publicly available BCI Competition IV, in which the subjects performed motor imagery (MI) tasks, and (2) the dataset created by healthy subjects of our laboratory performing motor movement (MO) tasks. The experimental results indicate that the brain data selected by the GA improve the recognition accuracy of the target CNN (TCNN) using pre-trained base CNN (BCNN). The improvement in accuracy is 11% and 4% for the BCI Competition IV and our laboratory datasets, respectively. In addition, the GA-selected training data reduce the CNN training time. The performance of the trained CNN, utilizing transfer learning, is tested for real-time control of a robot manipulator.

Real-time voice-controlled human machine interface system for wheelchairs implementation using Raspberry Pi

Real-time voice-controlled human machine interface system for wheelchairs implementation using Raspberry Pi

(Invited) ultrasensitive Low Cost Flexible Sensors and Bioinspired Sutureless Anastomosis Devices

I will present our efforts for investigating flexible electronic sensors for potential health monitoring applications and bioinspired sutureless anastomosis devices for easier, faster, and safer surgery.Highly sensitive flexible tactile sensors are of continuing interest for various applications including human–machine interface systems, internet of things, and wearable health monitoring devices. Current technologies for high sensitivity piezoresistive sensors tend to rely on costly materials and/or fabrication methods, limiting accessibility and scalability. Here, we report a facile sacrificial casting–etching method to synthesize nanoporous carbon nanotube/polymer composites for ultra-sensitive and low-cost piezoresistive pressure sensors. Our synthesis method overcomes the limitations of the traditional solution-dip-coating method for adhering nanoscale conductive materials to the nanoscale porous surface. Importantly, we show ultra-high sensitivity with a strain gauge factor over 300, which is ∼50 times higher than that of traditional CNT-based piezoresistive sensors. For practical tactile sensing applications, we demonstrate that the sensors can detect both gentle pressures (1 Pa–1 kPa) and low pressures (1 kPa–25 kPa) with a fraction of the cost. Our nanoporous polymer composite could contribute to expanding the scope of using nanocomposite sensors for medical applications from its easy tunability for sensing diverse range of tactile signals.Anastomosis, the surgical connection of adjacent tubular organs is a foundational surgical skill for many surgical procedures. The current standard of anastomosis is manually suturing two tubular structures together around an opening with fine sutures. However, it requires several years of surgical training, a long time (60 to 90 minutes per anastomosis) for procedure, and, at times, specialized equipment, facility and staffs. Thus, it is an expensive (up to $35,000 per procedure).Moreover, even in the hands of skilled surgeons, the anastomosis can be complicated by leakage or thrombosis. To address current challenges, we investigated a sutureless anastomosis device inspired by rose prickles. I will present our approach to design, fabricate, and test devices. Our device showed the failure force comparable to that of the handsewn suture (4.9 N) and no leakage up to 45 mL/min flow rate, well above the physiologic blood flow rate. Compared with handsewn anastomosis, the device resulted in minimum deformation of the anastomotic site. From in-vivo non-survival porcine studies, the device showed successful anastomosis (< 5 min per anastomosis) with no leaking for both arterial and venous anastomoses. We envision our findings can contribute to enabling easier, faster, and safer anastomosis, and making anastomotic techniques more accessible to a broad range of clinicians, researchers, and patients across the world.

Human Machine Interface System for Filling and Capping Process

In this research, bottle filling and capping processes are performed simultaneously in the pharmaceutical factory by using the PC based human machine interface (HMI) control system. In the traditional method of bottle filling process, the bottles are placed onto a conveyor and only one bottle is filled at a time. So, this method is time consuming and expensive as wide working place and many human labors is required. In this research, the proposed design can solve this problem without any complex control by using the HMI automation control system. The monitoring and running conditions in the packaging system are shown on the monitor of computer by using the composed of Visual Basic.Net and Mikro C. It also includes the volume of medical powder is limited by changing the size of filling funnel which is set up of shutter at the top and bottom of funnel. The entire system is more flexible and time saving. A prototype is implemented by using the DC motor, sensing devices; limit switches, peripheral interface controller (PIC) device and serial port communication. These control systems is very flexible, cost effective, space efficient and reduce complexity. PC based HMI control is used for monitoring the process.

Design of PLC control system for cascading tanks controlling

The article deals with the design of a PLC control system for controlling cascade tanks in the technological process of liquid mixing. The mixing process is carried out automatically according to information from individual sensors. The system also includes elements for manual control of the technological system. A human machine interface system was designed to operate the system. The simulation showed the correctness of the designed system.

Real-time voice-controlled human machine interface system for wheelchairs implementation using Raspberry Pi

Real-time voice-controlled human machine interface system for wheelchairs implementation using Raspberry Pi

Smart Contact Lenses for the New Era of IoT: Integrated Biosensors, Circuits, and Human–Machine Interface Systems

AbstractThe rapid development of the Internet of Things (IoT) technology endows some traditional devices with intelligent functions. Compared with traditional contact lenses used for correcting vision or beautifying the eyes, smart contact lenses (SCLs) are developed to monitor some physiological information of the eye. Sculls can be used to continuously monitor eye diseases non‐invasively in real‐time. As a personal electronic device, SCs can aid people in understanding their physical condition better without affecting their personal life. This review mainly discusses the development direction and problems associated with sculls from the perspective of the IoT. SCs consist of three main parts: biosensors, circuits, and human–machine interface (HMI) systems. These three parts correspond to the sensing, connection, and application layer of the IoT. Finally, the problems that may be encountered in the development of SCLs are discussed and potential solutions are proposed.

Human–machine interface-based wheelchair control using piezoelectric sensors based on face and tongue movements

Hand-free control of assistive mobility devices has been developed to serve people with movement disabilities at all levels. In this study, we demonstrate a human–machine interface (HMI) system that uses piezoelectric sensors to translate face and tongue movements. This study addresses two issues. First, we used six piezoelectric sensors to acquire muscular facial signals to observe the sensor positions and features during winking and tongue movements. Second, we verified the proposed HMI for online simulated wheelchair control. Twelve volunteers participated in the experiment. A maximum classification accuracy of 98.0% from the maximum and mean parameters could be achieved using the linear discriminant analysis and K-nearest neighbors classification algorithms. Using the proposed algorithm, command translation patterns for command translation reached more than 95% of the average classification accuracy with 0.5 s of the window for command creation. For online control of the simulated wheelchair, the results showed high efficiency based on the time condition. The combination of winking and tongue actions results in a steering time of the same magnitude as that of joystick-based control, which is less than twice the time of a joystick. Hence, the proposed system can be further implemented in a powered wheelchair for quadriplegic patients who retain control in the face or tongue muscles.

Review of Closed-Loop Brain–Machine Interface Systems From a Control Perspective

In recent years, brain–machine interface (BMI) technology has made great progress in controlling external devices and restoring motor function for people with disabilities. To better optimize BMI system performance, in this article, we summarize and describe a universal closed-loop BMI system framework and review the latest developments over the past ten years from a control perspective. First, the basic BMI systems with open-loop and closed-loop structures are introduced in chronological order. Second, the units of the universal closed-loop BMI system, i.e., the decoder, encoder, and auxiliary controller, are reviewed and summarized in terms of principles, categories, and algorithms. Finally, from research and practical perspectives, the importance of biomimetic brain models, great challenges, and future developments are discussed based on current progress. With this analysis of the universal closed-loop framework, this review can provide necessary theoretical guidance for the research and development of BMI systems.

A smart PLC-SCADA framework for monitoring petroleum products terminals in industry 4.0 via machine learning

The paper introduces a Programmable Logic Controller (PLC) / Human Machine Interface (HMI) system incorporated along with machine learning (ML) classifiers. Although some other studies have incorporated ML techniques to predict and control petroleum product terminals in terms of concentration, the proposed framework incorporates Add On Instruction (AOI) programming, PLC, and ML methods to automatically monitor petroleum products terminals. The framework adds an AOI in programming to achieve maximum usage of processor capabilities. Moreover, it uses AOI for programming in cooperation with the ladder diagram (LD). This leads to simplifying the LD graphical programming language, reducing the time of scanning, and making facilitate troubleshooting. The AOI is merged with ML to automate tank level detection and maintain good operational conditions and consequently protect these expensive essential assets. The introduced framework consists of three stages. The first stage is the PLC programming phase where the PLC is created using Add-On instructions. Next, HMI graphic displays are drawn and linked to the PLC tags in the following stage. During the third stage, the actual process readings are applied to the system based on ML algorithms to test its functionality. The proposed system results indicates a reduction in the LD number, highest program size, and maximum time of scanning. The results indicate that the AOI can help to trace the program more easily in fault situations. Besides, additional program instructions could reduce processor memory, system construction costs, and upgrade projects.

Farming‐Inspired Continuous Fabrication of Grating Flexible Transparent Film with Anisotropic Conductivity

AbstractFlexible transparent conductive films (FTCFs) are critically important for the emerging flexible electronic devices. To this end, FTCFs with conductive gratings have drawn more interests in the past decades. However, there are great challenges to fabricate such FTCFs containing conductive gratings in a facile way. Borrowing the idea from traditional farming, a homemade roll‐to‐roll plowing technique combined with blade coating is proposed to fabricate silver grating FTCFs with anisotropic conductivity in a continuous, cost‐effective, and environmental friendly way. The as‐prepared FTCFs exhibit satisfied optical and electrical properties (i.e., transmittance is 78.2% and sheet resistance is 3.1 Ω □–1). After coated with thin polyvinyl butyral layer through hot‐pressing, it shows a higher transmittance of 84.2% and remarkable mechanical flexibility. Based on the FTCFs with anisotropic conductivity, flexible transparent heater and capacitive keyboard are successfully fabricated. Furthermore, based on such flexible capacitive keyboard, smart wireless controlled human–machine interface system is developed to wirelessly control tablet personal computer working and quadrotor flying. This study provides a promising strategy to fabricate high‐performance FTCFs with anisotropic conductivity via a continuous, cost‐effective, and environmental friendly way.

Personalized voice activated grasping system for a robotic exoskeleton glove

This paper proposes a novel human machine interface (HMI) and electronics system design to control a rehabilitation robotic exoskeleton glove. Such system can be activated with the user’s voice, take voice commands as input, recognize the command and perform biometric authentication in real-time with limited computing power, and execute the command on the exoskeleton. The electronics design is a stand-alone plug-and-play modulated design independent of the exoskeleton design. This personalized voice activated grasping system achieves better wearability, lower latency, and improved security than any existing exoskeleton glove control system.

Algorithm and hardware considerations for real-time neural signal on-implant processing

Objective. Various on-workstation neural-spike-based brain machine interface (BMI) systems have reached the point of in-human trials, but on-node and on-implant BMI systems are still under exploration. Such systems are constrained by the area and battery. Researchers should consider the algorithm complexity, available resources, power budgets, CMOS technologies, and the choice of platforms when designing BMI systems. However, the effect of these factors is currently still unclear. Approaches. Here we have proposed a novel real-time 128 channel spike detection algorithm and optimised it on microcontroller (MCU) and field programmable gate array (FPGA) platforms towards consuming minimal power and memory/resources. It is presented as a use case to explore the different considerations in system design. Main results. The proposed spike detection algorithm achieved over 97% sensitivity and a smaller than 3% false detection rate. The MCU implementation occupies less than 3 KB RAM and consumes 31.5 µW ch−1. The FPGA platform only occupies 299 logic cells and 3 KB RAM for 128 channels and consumes 0.04 µW ch−1. Significance. On the spike detection algorithm front, we have eliminated the processing bottleneck by reducing the dynamic power consumption to lower than the hardware static power, without sacrificing detection performance. More importantly, we have explored the considerations in algorithm and hardware design with respect to scalability, portability, and costs. These findings can facilitate and guide the future development of real-time on-implant neural signal processing platforms.

PURWARUPA SISTEM PENGENDALIAN DAN AKUISISI DATA PADA RUMAH CERDAS BERBASIS SMARTPHONE

Internet of Things (IoT) is the latest technology that has an extraordinary role in improving the quality of life and developing the world economy. IoT also known at community widely which is used to support global connectivity in various sectors, one of them is the smart home. Smart home is a new way in home appliances or equipment such as a lamp, air conditioner, heater and gate which is that equipment can be controlled remotely via the internet with the Human Machine Interface (HMI) system. On this final project, a prototype control and data acquisition system is designed for smart home based on smartphone. The system is designed using ATMega2560 and ESP8266 microcontrollers along with supporting sensors, while the HMI system is designed based on Android which can perform the functions of Supervisory Control and Data Acquisition (SCADA) namely measuring physical quantities (telemetering), notification of alarming systems (tele-signaling) and controlling home appliances easily and efficiently via the internet (tele-controlling). Sending and receiving data by the microcontroller to the server has an interval of 889 ms. The success rate and communication speed are influenced by internet connection and data traffic on the server. The data obtained in the telemetering system corresponds to the real situation of the smart home prototype. Tele-controlling and tele-signaling systems work well when operated.

Research on algorithms for control design of human–machine interface system using ML

Research on algorithms for control design of human–machine interface system using ML

Human–Machine Interaction in Driving Assistant Systems for Semi-Autonomous Driving Vehicles

Currently, the existing vehicle-centric semi-autonomous driving modules do not consider the driver’s situation and emotions. In an autonomous driving environment, when changing to manual driving, human–machine interface and advanced driver assistance systems (ADAS) are essential to assist vehicle driving. This study proposes a human–machine interface that considers the driver’s situation and emotions to enhance the ADAS. A 1D convolutional neural network model based on multimodal bio-signals is used and applied to control semi-autonomous vehicles. The possibility of semi-autonomous driving is confirmed by classifying four driving scenarios and controlling the speed of the vehicle. In the experiment, by using a driving simulator and hardware-in-the-loop simulation equipment, we confirm that the response speed of the driving assistance system is 351.75 ms and the system recognizes four scenarios and eight emotions through bio-signal data.