Human-machine Interface Applications Research Articles

Recent progress of optical tactile sensors: A review

Over the past decade, flexible electronics has become a cutting-edge technology owing to its broad applications in robotics, health monitoring, and human-machine interface. The recent development of flexible electrical tactile sensor triggers the research of optical tactile sensors due to their potentials in high-sensitivity, high-precision, low power consumption, and anti-electromagnetic interference. Here, this review mainly discusses the advances and applications of silica fiber, polymer fiber, hydrogel, and micro/nanofiber based tactile sensors, specifically, focusing on the latest research results and on the necessary technologies to enable the health monitoring, robotics, and human-machine interface applications. Furthermore, future outlooks in developing skin-like optical tactile sensors are also presented.

Highly stretchable and strain sensitive MXene/MXene:MWCNTs@TPU fiber with hierarchical conductive layers and porous elastic core structure

Fiber flexible strain sensors have the characteristics of small volume, light weight, good flexibility, easy deformation/recovery, and easy weaving. They can conform to various complex surfaces and endure various deformations, such as twisting and bending. Furthermore, the simple preparation process of fiber sensors lowers the production cost significantly. These features make them promising candidates for applications in motion monitoring, human-machine interface, and flexible intelligent products. However, the low sensitivity and narrow sensing range of existing sensors are still major challenges for their further development. To address these challenges, this paper proposes a MXene/MXene:MWCNTs@TPU fiber sensor, which adopts a two-step pre-stretching strategy to modify the porous TPU fiber surface with MXene and MWCNTs. The graded porous TPU fibers obtained by wet spinning can achieve 805 % stretching, providing a reliable substrate support for the fiber sensor, the combination of multi-dimensional conductive materials and the introduction of cracks and wrinkles enhance the sensitivity 277 and sensing range 510 % of the fiber sensor. The fiber sensor can detect various human motions and gestures by directly attaching to the skin or weaving into textiles, and can also be applied to information encryption transmission, demonstrating its huge potential in human health monitoring, intelligent flexible electronic products and encrypted communication.

A Low‐Cost and Do‐It‐Yourself Pressure Sensor Enable Human Motion Detection and Human–Machine Interface Applications

AbstractIn this work, cost‐effective and do‐it‐yourself capacitive pressure sensors are fabricated using readily available commercial components. The sensors are created in a single‐step process ‐by simply applying electrically conductive paint onto both sides of a porous melamine sponge. These sensors exhibit a wide‐range pressure sensing capability, spanning from 10 Pa to 100 kPa. The sensors showcase an impressively low limit of detection, detecting pressures as low as 10 Pa, and exhibit a moderate response time of 123 ms. Moreover, the sensors display remarkable repeatability and stability over 10 000 loading and unloading cycles without experiencing fatigue. Notably, these exceptional qualities come at an exceptionally low material cost, with the sensor measuring 20 × 20 × 2 mm. To showcase their potential applications, the fabricated sensors are successfully employed in real‐time human motion detection, proximity detection, and wearable keyboard applications.

Novel near E-Field Topography Sensor for Human-Machine Interfacing in Robotic Applications.

This work investigates a new sensing technology for use in robotic human-machine interface (HMI) applications. The proposed method uses near E-field sensing to measure small changes in the limb surface topography due to muscle actuation over time. The sensors introduced in this work provide a non-contact, low-computational-cost, and low-noise method for sensing muscle activity. By evaluating the key sensor characteristics, such as accuracy, hysteresis, and resolution, the performance of this sensor is validated. Then, to understand the potential performance in intention detection, the unmodified digital output of the sensor is analysed against movements of the hand and fingers. This is done to demonstrate the worst-case scenario and to show that the sensor provides highly targeted and relevant data on muscle activation before any further processing. Finally, a convolutional neural network is used to perform joint angle prediction over nine degrees of freedom, achieving high-level regression performance with an RMSE value of less than six degrees for thumb and wrist movements and 11 degrees for finger movements. This work demonstrates the promising performance of this novel approach to sensing for use in human-machine interfaces.

High-precision strategy for piezoelectric characterization of nano/microwire

Investigating piezoelectricity of nano/microwire is important for their applications in nanogenerator, actuator, sensor and many more. Up to now, accurately probing the piezoelectric constant of nano/microwire along its axial direction has remained a challenge. Here, we propose a strategy that can high-precisely characterize the piezoelectric constant of nano/microwire along its axial direction through constructing a lateral structure with short connection of two ends of nano/microwire as one electrode and probe tip as another electrode, making the nano/microwire not buckle and its length unchanged during measurements. Theoretical analysis and simulations show that this strategy has ultra-high precision, in which the error can be as low as 2% (aspect ratio ∼100), 0.3% (∼500) and less than 0.2% (∼1000). Using this strategy, the measured piezoelectric constant d33 of ZnO microwires is 12.31 ± 0.57 pm/V, highly consistent with widely accepted values in ZnO bulks and films, further verifying the correctness and accuracy of this strategy. This simple, universal and high-precision strategy contributes to the piezoelectric characterization of nano/microwires and their applications in energy harvesting, human-machine interfacing, active electronics/optoelectronics and many more.

Multi-Channel Soft Dry Electrodes for Electrocardiography Acquisition in the Ear Region.

In-ear acquisition of physiological signals, such as electromyography (EMG), electrooculography (EOG), electroencephalography (EEG), and electrocardiography (ECG), is a promising approach to mobile health (mHealth) due to its non-invasive and user-friendly nature. By providing a convenient and comfortable means of physiological signal monitoring, in-ear signal acquisition could potentially increase patient compliance and engagement with mHealth applications. The development of reliable and comfortable soft dry in-ear electrode systems could, therefore, have significant implications for both mHealth and human-machine interface (HMI) applications. This research evaluates the quality of the ECG signal obtained with soft dry electrodes inserted in the ear canal. An earplug with six soft dry electrodes distributed around its perimeter was designed for this study, allowing for the analysis of the signal coming from each electrode independently with respect to a common reference placed at different positions on the body of the participants. An analysis of the signals in comparison with a reference signal measured on the upper right chest (RA) and lower left chest (LL) was performed. The results show three typical behaviors for the in-ear electrodes. Some electrodes have a high correlation with the reference signal directly after inserting the earplug, other electrodes need a settling time of typically 1-3 min, and finally, others never have a high correlation. The SoftPulseTM electrodes used in this research have been proven to be perfectly capable of measuring physiological signals, paving the way for their use in mHealth or HMI applications. The use of multiple electrodes distributed in the ear canal has the advantage of allowing a more reliable acquisition by intelligently selecting the signal acquisition locations or allowing a better spatial resolution for certain applications by processing these signals independently.

A skin-conformal and breathable humidity sensor for emotional mode recognition and non-contact human-machine interface

Noncontact humidity sensor overcomes the limitations of its contact sensing counterparts, including mechanical wear and cross infection, which becomes a promising candidate in healthcare and human-machine interface application. However, current humidity sensors still suffer the ubiquitous issue of uncomfortable wear and skin irritation hindering the long-term use. In this study, we report a skin-conformal and breathable humidity sensor assembled by anchoring MXenes-based composite into electrospun elastomer nanofibers coated with a patterned electrode. This composite is highly sensitive to the water molecules due to its large specific surface area and abundant water-absorbing hydroxyl groups, while the elastomeric nanofibers provide an ultrathin, highly flexible, and permeable substrate to support the functional materials and electrodes. This sensor presents not only excellent air permeability (0.078 g cm−2 d−1), high sensitivity (S = 704), and fast response/recovery (0.9 s/0.9 s), but also high skin conformability and biocompatibility. Furthermore, this humidity sensor is confirmed to realize the recognition of motional states and emotional modes, which provides a way for the advanced noncontact human-machine interface.

Synergistic advancements in high-performance flexible capacitive pressure sensors: structural modifications, AI integration, and diverse applications.

The development of flexible pressure sensors for monitoring human motion and physiological signals has attracted extensive scientific research. However, achieving low monitoring limits, a wide detection range, large bending stresses, and excellent mechanical stability simultaneously remains a serious challenge. With the aim of developing a high-performance capacitive pressure sensor (CPS), this paper introduces the successful preparation of a single-walled carbon nanotube (SWNT)/polydimethylsiloxane (S-PDMS) composite dielectric with a foam-like structure (high permittivity and low elasticity modulus) and MXene/SWNT (S-MXene) composite film electrodes with a micro-crumpled structure. The above structurally modified CPS (SMCPS) demonstrated an excellent response output during pressure loading, achieving a wide pressure detection range (up to 700 kPa), a low detection limit (16.55 Pa), fast response/recovery characteristics (48/60 ms), enhanced sensitivity across a wide pressure range, long-term stability under repeated heavy loading and unloading (40 kPa, >2000 cycles), and reliable performance under various temperature and humidity conditions. The SMCPS demonstrated a precise and stable capacitive response in monitoring subtle physiological signals and detecting motion, owing to its unique electrode structure. The flexible device was integrated with an Internet of Things module to create a smart glove system that enables real-time tracking of dynamic gestures. This system demonstrates exceptional performance in gesture recognition and prediction with artificial intelligence analysis, highlighting the potential of the SMCPS in human-machine interface applications.

A Novel Approach to Surface EMG-based Gesture Classification Using a Vision Transformer Integrated with Convolutive Blind Source Separation.

A robust pattern recognition framework is required for ideal real-time human-machine interface (HMI) applications. Convolutional neural networks and recurrent neural networks have been widely used for the classification of gestures based on electromyography (EMG), but few studies have demonstrated the effectiveness of using a vision transformer for this purpose. Additionally, the accuracy achieved is influenced by the efficacy of the preprocessing pipeline. This study assessed ViT with and without an attention mechanism for precise motor intent decoding by investigating various input features and integrating convolutive blind source separation (BSS) preprocessing. All investigations were carried out with two open-access high-density surface EMG datasets of 34 and 21 hand gestures recorded from 20 and 5 healthy subjects respectively. Integration of centering and optimal extension factors resulted in better performance with raw input. However, spatial whitening increased the model's sensitivity to noise. The best-performing BSS-integrated convolution vision transformer model (BSS-CViT) model yielded an accuracy of 96.61% and 91.98% on test datasets one and two. This is a promising result for future studies in real-time HMI applications. The code implementation results reported in this study are available on GitHub. https://github.com/deremustapha/BSS-ViT.

A triboelectric nanogenerator based wide range selfpowered flexible pressure sensor

Self-powered flexible pressure sensors are needed in applications such as human-machine interaction. Here, we present a triboelectric nanogenerator (TENG) based self-powered flexible pressure sensor capable of operating over a wide-range of pressures (3.2 - 1176 kPa). The sensor exhibits excellent sensitivities in various regions of the pressure range, i.e., 3.16, 0.023 and 0.031 V/kPa in the low (1-10 kPa), medium-high (10-500 kPa) and ultra-high (>500 kPa) pressure regimes respectively. Stable and repeatable TENG responses are achieved for all three pressure regimes, and this is due to the way the real contact area at the TENG interface varies with contact pressure. These results highlight the potential for TENGs in a wide range of applications such as: detecting pressure in wearable devices, human-machine interface, biomedical and automotive applications. As a proof-of-concept, we have demonstrated the use of the presented device in applications such as detection of human and robot finger tapping, collection of human gait information, and detection of impact forces.

A flexible dual-mode triboelectric sensor for strain and tactile sensing toward human-machine interface applications

In recent years, there has been significant progress in the development of flexible sensors for human-machine interfaces. However, the triboelectric principle, especially which based on sliding mode, is relative rarely applied to flexible strain sensors. Therefore, a novel triboelectric flexible stretchable sensor has been designed with a simple structure, convenient fabrication process, and effective response to various strain signals (Within the range of 0%−200%, the average sensitivity is 49.07 mV per 1% strain). This sensor operates in a new triboelectric working mode: variable charge density mode (VCD mode). Furthermore, the sensor can also be utilized for tactile sensing in vertical contact-separation mode without any structural modifications. Based on this strain-tactile dual-mode flexible triboelectric sensor (STDFTS), its applications for human-machine interface have been explored: gesture recognition (recognition accuracy of five gestures reaches 99.4%) based on tactile sensing and mouse replacement based on dual working modes. The STDFTS proposed in this article provides a new alternative solution for human-machine interfaces, greatly expanding the application scenarios of triboelectricity. The sensor's innovative design and dual-mode functionality open up new avenues for research and development in the field of triboelectric nanogenerator, flexible sensors and human-machine interaction.

A comprehensive survey of automatic dysarthric speech recognition

The need for automated speech recognition has expanded as a result of significant industrial expansion for a variety of automation and human-machine interface applications. The speech impairment brought on by communication disorders, neurogenic speech disorders, or psychological speech disorders limits the performance of different artificial intelligence-based systems. The dysarthric condition is a neurogenic speech disease that restricts the capacity of the human voice to articulate. This article presents a comprehensive survey of the recent advances in the automatic dysarthric speech recognition (DSR) using machine learning (ML) and deep learning (DL) paradigms. It focuses on the methodology, database, evaluation metrics, and major findings from the study of previous approaches. From the literature survey it provides the gaps between exiting work and previous work on DSR and provides the future direction for improvement of DSR. The performance of the various machine and DL schemes is evaluated for the DSR on UASpeech dataset based on accuracy, precision, recall, and F1-score. It is observed that the DL based DSR schems outperforms the ML based DSR schemes.

Flexible Pressure Sensors in Human-Machine Interface Applications.

Flexible sensors are highly flexible, malleable, and capable of adapting todifferent shapes, surfaces, and environments, which opens a wide range ofpotential applications in the field of human-machine interface (HMI). Inparticular, flexible pressure sensors as a crucial member of the flexiblesensor family, are widely used in wearable devices, health monitoringinstruments, robots and other fields because they can achieve accuratemeasurement and convert the pressure into electrical signals. The mostintuitive feeling that flexible sensors bring to people is the change ofhuman-machine interface interaction, from the previous rigid interaction suchas keyboard and mouse to flexible interaction such as smart gloves, more inline with people's natural control habits. Many advanced flexible pressuresensors have emerged through extensive research and development, and to adaptto various fields of application. Researchers have been seeking to enhanceperformance of flexible pressure sensors through improving materials, sensingmechanisms, fabrication methods, and microstructures. This paper reviews the flexible pressure sensors in HMI in recent years, mainlyincluding the following aspects: current cutting-edge flexible pressuresensors; sensing mechanisms, substrate materials and active materials; sensorfabrication, performances, and their optimization methods; the flexiblepressure sensors for various HMI applications and their prospects.

Survey of Automatic Dysarthric Speech Recognition

The need for automated speech recognition has expanded as a result of significant industrial expansion for a variety of automation and human-machine interface applications. The speech impairment brought on by communication disorders, neurogenic speech disorders, or psychological speech disorders limits the performance of different artificial intelligence-based systems. The dysarthric condition is a neurogenic speech disease that restricts the capacity of the human voice to articulate. This article presents a comprehensive survey of the recent advances in the automatic Dysarthric Speech Recognition (DSR) using machine learning and deep learning paradigms. It focuses on the methodology, database, evaluation metrics and major findings from the study of previous approaches. From the literature survey it provides the gaps between exiting work and previous work on DSR and provides the future direction for improvement of DSR.

Recent Developments and Future Directions of Wearable Skin Biosignal Sensors

AbstractThis review article explores the transformative advancements in wearable biosignal sensors powered by machine learning, focusing on four notable biosignals: electrocardiogram (ECG), electromyogram (EMG), electroencephalogram (EEG), and photoplethysmogram (PPG). The integration of machine learning with these biosignals has led to remarkable breakthroughs in various medical monitoring and human–machine interface applications. For ECG, machine learning enables automated heartbeat classification and accurate disease detection, improving cardiac healthcare with early diagnosis and personalized interventions. EMG technology, combined with machine learning, facilitates real‐time prediction and classification of human motions, revolutionizing applications in sports medicine, rehabilitation, prosthetics, and virtual reality interfaces. EEG analysis powered by machine learning goes beyond traditional clinical applications, enabling brain activity understanding in psychology, neurology, and human–computer interaction, and holds promise in brain–computer interfaces. PPG, augmented with machine learning, has shown exceptional progress in diagnosing and monitoring cardiovascular and respiratory disorders, offering non‐invasive and accurate healthcare solutions. These integrated technologies, powered by machine learning, open new avenues for medical monitoring and human–machine interaction, shaping the future of healthcare.

A near-infrared light-promoted self-healing photothermally conductive polycarbonate elastomer based on Prussian blue and liquid metal for sensors

Composite elastomers with elasticity, conductivity, and self-healing properties have gained tremendous interest due to the imperative demands in the fields of stretchable electronics and soft robotics. However, the self-healing performance and the amount of filler are contradictory. Herein, a new conductive self-healing composite elastomer is developed by uniformly dispersing EGaIn droplets and Prussian blue nanoparticles (PBNPs) in a bran-new elastomer which cross-linked the linear polymer that obtained by ring-opening polymerization of trimethylene carbonate and 5-methyl-5-carboxytrimethylene carbonate initiated by polyethylene glycol by aluminum chloride. As confirmed by FT-IR and XPS, the cross-linking network of the composite elastomer is composed of hydrogen bonds and coordination bonds sheared between aluminum and carboxyl groups, and the coordination process was revealed by DFT calculations. This elastomer exhibits excellent light-to-heat conversion properties, thermal conductivity (1.207 W/mK), electrical conductivity (202.34 S·m−1), and good tensile properties that meet application requirements. The good photothermal performance enables the elastomer to self-heal rapidly under NIR irradiation (90.3 %), and accelerate the shape recovery of the elastomer. As a sensor, the elastomer demonstrates good sensitivity, capable of monitoring human movements and recognizing handwriting. This self-healable conductive elastomer has significant potential in the fields of damage-resistant flexible sensors and human-machine interface applications.

Novel Muscle Sensing by Radiomyography (RMG) and Its Application to Hand Gesture Recognition.

Conventional electromyography (EMG) measures the continuous neural activity during muscle contraction, but lacks explicit quantification of the actual contraction. Mechanomyography (MMG) and accelerometers only measure body surface motion, while ultrasound, CT-scan and MRI are restricted to in-clinic snapshots. Here we propose a novel radiomyography (RMG) for continuous muscle actuation sensing that can be wearable or touchless, capturing both superficial and deep muscle groups. We verified RMG experimentally by a wearable forearm sensor for hand gesture recognition (HGR). We first converted the sensor outputs to the time-frequency spectrogram, and then employed the vision transformer (ViT) deep learning network as the classification model, which can recognize 23 gestures with an average accuracy up to 99% on 8 subjects. By transfer learning, high adaptivity to user difference and sensor variation were achieved at an average accuracy up to 97%. We further extended RMG to monitor eye and leg muscles and achieved high accuracy for eye movement and body posture tracking. RMG can be used with synchronous EMG to derive stimulation-actuation waveforms for many potential applications in kinesiology, physiotherapy, rehabilitation, and human-machine interface.

Highly Flexible, Self-Bonding, Self-Healing, and Conductive Soft Pressure Sensors Based on Dicarboxylic Cellulose Nanofiber Hydrogels.

Conductive hydrogels have gained a great deal of interest in the flexible electronics industry because of their remarkable inherent properties. However, a significant challenge remains for balancing hydrogel's conductivity, self-healing, and strength properties. Herein, double network ionic hydrogels were fabricated by concurrently introducing borax into dicarboxylic cellulose nanofiber (DCNFs) and polyacrylamide (PAM) hydrogels. The incorporation of borax provided a superabsorbent feature to the PAM/DCNF hydrogels (without borax) with the equilibrium water absorption rate increased from 552 to 1800% after 42 h. The compressive strength of the prepared hydrogel was 935 kPa compared to 132 kPa for the PAM hydrogel, with high cycling stability (stable after 1000 compression cycles with 50% strain). The hydrogel pressure sensor had a very sensitive response (gauge factor = 1.36) in the strain range from 10 to 80%, which made it possible to detect mechanical motion accurately and reliably. The developed hydrogels with high-performance, environmentally friendly properties are promising for use in future artificial skin and human-machine interface applications.

Finger gesture recognition with smart skin technology and deep learning

Finger gesture recognition (FGR) was extensively studied in recent years for a wide range of human-machine interface applications. Surface electromyography (sEMG), in particular, is an attractive, enabling technique in the realm of FGR, and both low and high-density sEMG were previously studied. Despite the clear potential, cumbersome electrode wiring and electronic instrumentation render contemporary sEMG-based finger gestures recognition to be performed under unnatural conditions. Recent developments in smart skin technology provide an opportunity to collect sEMG data in more natural conditions. Here we report on a novel approach based on soft 16 electrode array, a miniature and wireless data acquisition unit and neural network analysis, in order to achieve gesture recognition under natural conditions. FGR accuracy values, as high as 93.1%, were achieved for 8 gestures when the training and test data were from the same session. For the first time, high accuracy values are also reported for training and test data from different sessions for three different hand positions. These results demonstrate an important step towards sEMG based gesture recognition in non-laboratory settings, such as in gaming or Metaverse.

Spider-web inspired self-healing, adhesive, injectable conductive hydrogel for human motion monitoring and shape recognition

Conductive hydrogels have received significant attention in fabricating wearable strain sensors, electronic skins, and human-machine interface applications, but suffer the poor cycling capacity and hard to be applied in complex sensing applications. Herein a spider web-inspired poly (stearyl methacrylate-co-acrylic acid) and polyvinyl alcohol interpenetrating network conductive hydrogel was synthesized via one-step in-situ polymerization to address these challenges aforementioned. The as-obtained hydrogels exhibit good mechanical strength (111 kPa stress), high stretchability (1029% strain), strong adhesiveness, high conductivity, injectability, and rapid self-healing performance (the healing efficiency of 87.3% at room temperature within 4 h). Attributing to excellent self-healing properties, the hydrogels were easily reprocessed four times at room temperature without losing their intrinsic characteristics. Additionally, high strain sensitivity (GF = 4.28 in the wide strain-sensing range 1–500%) and real time monitoring large/tiny human motions have been also realized. Most importantly, a assembled sensing array has been demonstrated via the injectable behaviour to detect the mass and contact shape of the target We believe that this study will definitely broaden the practical applications of a new generation of hydrogel-based wearable strain sensor.