Applications In Human-machine Interaction Research Articles

Facile Electret-Based Self-Powered Soft Sensor for Noncontact Positioning and Information Translation.

Noncontact sensors have demonstrated significant potential in human-machine interactions (HMIs) in terms of hygiene and less wear and tear. The development of soft, stable, and simply structured noncontact sensors is highly desired for their practical applications in HMIs. This work reports on electret-based self-powered noncontact sensors that are soft, transparent, stable, and easy to manufacture. The sensors contain a three-layer structure with a thickness of 0.34 mm that is fabricated by simply stacking a polymeric electret layer, an electrode layer, and a substrate layer together. The fabricated sensors show high charge-retention capability, keeping over 98% of the initial surface potential even after 90 h, and can accurately and repeatedly sense external approaching objects with impressive durability. The intensity of the detected signal shows a strong dependence on the distance between the object and the sensor, capable of sensing a distance as small as 2 mm. Furthermore, the sensors can report stable signals in response to external objects over 3000 cycles. By virtue of the signal dependence on distance, an intelligent noncontact positioning system is developed that can precisely detect the location of an approaching object. Finally, by integrating with eyeglasses, the transparent sensor successfully captures the movements of blinks for information translation. This work may contribute to the development of stable and easily manufactured noncontact soft sensors for HMI applications, for instance, assisting with communication for locked-in syndrome patients.

Amharic spoken digits recognition using convolutional neural network

Spoken digits recognition (SDR) is a type of supervised automatic speech recognition, which is required in various human–machine interaction applications. It is utilized in phone-based services like dialing systems, certain bank operations, airline reservation systems, and price extraction. However, the design of SDR is a challenging task that requires the development of labeled audio data, the proper choice of feature extraction method, and the development of the best performing model. Even if several works have been done for various languages, such as English, Arabic, Urdu, etc., there is no developed Amharic spoken digits dataset (AmSDD) to build Amharic spoken digits recognition (AmSDR) model for the Amharic language, which is the official working language of the government of Ethiopia. Therefore, in this study, we developed a new AmSDD that contains 12,000 utterances of 0 (Zaero) to 9 (zet’enyi) digits which were recorded from 120 volunteer speakers of different age groups, genders, and dialects who repeated each digit ten times. Mel frequency cepstral coefficients (MFCCs) and Mel-Spectrogram feature extraction methods were used to extract trainable features from the speech signal. We conducted different experiments on the development of the AmSDR model using the AmSDD and classical supervised learning algorithms such as Linear Discriminant Analysis (LDA), K-Nearest Neighbors (KNN), Support Vector Machine (SVM), and Random Forest (RF) as the baseline. To further improve the performance recognition of AmSDR, we propose a three layers Convolutional Neural Network (CNN) architecture with Batch normalization. The results of our experiments show that the proposed CNN model outperforms the baseline algorithms and scores an accuracy of 99% and 98% using MFCCs and Mel-Spectrogram features, respectively.

Ultrasensitive Biomimetic Skin with Multimodal and Photoelectric Dual-Signal Sensing.

Mimicking biological skin enabling direct, intelligent interaction between users and devices, multimodal sensing with optical/electrical (OE) output signals is urgently required. Owing to this, this work aims to logically design a stretchable OE biomimetic skin (OE skin), which can sensitively sense complex external stimuli of pressure, strain, temperature, and localization. The OE skin consists of elastic thin polymer-stabilized cholesteric liquid crystal films, an ion-conductive hydrogel layer, and an elastic protective membrane formed with thin polydimethylsiloxane. The as-designed OE skin exhibits customizable structural color on demand, good thermochromism, and excellent mechanochromism, with the ability to extend the full visible spectrum, a good linearity of over 0.99, fast response speed of 93 ms, and wide temperature range of 119 °C. In addition, the conduction resistance variation of ion-conductive hydrogel exhibits excellent sensing capabilities under pressure, stretch, and temperature, endowing a good linearity of 0.99998 (stretching from 0 to 150%) and high thermal sensitivity of 0.86% per °C. Such an outstanding OE skin provides design concepts for the development of multifunctional biomimetic skin used in human-machine interaction and can find wide applications in intelligent wearable devices and human-machine interactions.

Antifreeze protein-based ultrastretchable and highly sensitive hydrogel for human-machine interaction

Intelligent and convenient human–computer interfaces realize efficient and close cooperation between people and the digital world. As windows of human–machine interaction (HMI), sensors directly determine the ability and quality of acquired signals and the utilization potential of HMI systems. However, it still remains challenging to develop antifreeze hydrogel sensors with freeze-resistant properties while avoiding the loss of other properties as much as possible to achieve stable HMI applications in low-temperature environments. Here, we present an Agar/PAAm dual-network hydrogel (Na+-Agar/PAAm-AFP) doped with both antifreeze protein (AFP) and sodium chloride (NaCl), which exhibits excellent antifreeze capability (−20 °C). The hydrogel integrates transparency (>90%), ultra stretchability (up to 2100%), good adhesion properties to metal, glass, and plastic, high electrical conductivity (3.41 S m−1), high sensitivity (GF = 10.23), and non-irritating to the skin. The hydrogel can be used for monitoring daily activities and glove-based HMI systems for intelligent control. In addition, based on this hydrogel, we have successfully developed an interactive system for real-time information transfer over long distances. It is expected to provide better health management monitoring in several fields and also help to advance the development of innovative and IoT technologies.

Kirigami-Shaped Dual-Functional Strain and Temperature Sensors for Monitoring Body Movements and Hyperthermia Toward Physiotherapy Applications

Multifunctional sensors find widespread applications in wearable healthcare, Internet-of-Things, environmental monitoring, and human-machine interactions, yet they suffer from reduced stretchability, durability, wearability, and enhanced signal crosstalk. This work presents a new kirigami-shaped dual-functional patch that integrates both strain and temperature sensors in a single platform. The kirigami architecture allows out-of-plane deformation of the sensors enabling them to conform to different body parts. The sensing regions are functionalized with gold nanoparticle-decorated multi-walled carbon nanotubes that provide highly sensitive and repeatable performance. The strain sensor has a gauge factor of around 220 at 0.12% bending strain and the temperature sensor shows a sensitivity of 0.0278 kΩ °C <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-1</sup> . Additionally, the sensor demonstrates an ultra-high stretchability for an applied tensile strain of up to 180%. The sensors can precisely monitor the dynamic bending of different joints in the body and temperature as well as decouple the strain and temperature measurements in response to dual stimuli. Furthermore, the sensors are interfaced with an Internet-of-Things platform that allows wireless transmission of body motion and temperature measurements to the cloud. The versatile nature of the sensing patch holds great promise for continuous monitoring of the progress in physiotherapy so that recovery can be tracked in real-time and appropriate intervention measures can be applied.

Wafer-Recyclable, Eco-Friendly, and Multiscale Dry Transfer Printing by Transferable Photoresist for Flexible Epidermal Electronics.

Flexible electronics have been of great interest in the past few decades for their wide-ranging applications in health monitoring, human-machine interaction, artificial intelligence, and biomedical engineering. Currently, transfer printing is a popular technology for flexible electronics manufacturing. However, typical sacrificial intermediate layer-based transfer printing through chemical reactions results in a series of challenges, such as time consumption and interface incompatibility. In this paper, we have developed a time-saving, wafer-recyclable, eco-friendly, and multiscale transfer printing method by using a stable transferable photoresist. Demonstration of photoresist with various, high-resolution, and multiscale patterns from the donor substrate of silicon wafer to different flexible polymer substrates without any damage is conducted using the as-developed dry transfer printing process. Notably, by utilizing the photoresist patterns as conformal masks and combining them with physical vapor deposition and dry lift-off processes, we have achieved in situ fabrication of metal patterns on flexible substrates. Furthermore, a mechanical experiment has been conducted to demonstrate the mechanism of photoresist transfer printing and dry lift-off processes. Finally, we demonstrated the application of in situ fabricated electrode devices for collecting electromyography and electrocardiogram signals. Compared to commercially available hydrogel electrodes, our electrodes exhibited higher sensitivity, greater stability, and the ability to achieve long-term health monitoring.

Robust Fiber-Shaped Flexible Temperature Sensors for Safety Monitoring with Ultrahigh Sensitivity.

Flexible temperature sensors capable of detecting and transmitting temperature data from the human body, environment, and electronic devices hold significant potential for applications in electronic skins, human-machine interactions, and disaster prevention systems. Nonetheless, fabricating flexible temperature sensors with exceptional sensing performance remains a formidable task, primarily due to the intricate process of constructing an intrinsically flexible sensing element with high sensitivity. In this study, a facile in situ two-step synthetic method is introduced for fabricating flexible fiber-shaped NiO/carbon nanotube fiber (CNTF) composites. The resulting NiO/CNTF flexible temperature sensors demonstrate outstanding deformability and temperature sensing characteristics, encompassing a broad working range (-15 to 60°C) and high sensitivity (maximum TCR of -20.2%°C-1 and B value of 3332 K). Importantly, the mechanical and thermal behaviors of the sensor in various application conditions are thoroughly examined using finite element analysis simulations. Moreover, the temperature sensors can effectively capture diverse thermal signals in wearable applications. Notably, a temperature monitoring and warning system is developed to prevent fire accidents resulting from abnormal thermal runaway in electronic devices.

Flexible capacitive pressure sensors and their applications in human-machine interactions

Flexible pressure sensors offer more significant advantages than rigid sensors in various applications, including human-computer interaction, medical health, and robot touch. However, this also places more stringent demands on the materials used. For example, the materials must be thin and soft enough to fit onto the surface of the human skin or be implanted into the body while also exhibiting good biocompatibility and matching the mechanical properties of biological tissues. Regarding device performance, flexible pressure sensor design primarily focuses on enhancing sensitivity, response time, detection limit, and stability. Researchers have recently been expanding these sensors' pressure response range, pressure resolution, spatial resolution, and tensile performance, which will further broaden their potential applications. This review provides an overview of the recent classification of flexible pressure sensors, covering their sensing principle, sensing performance, and future application prospects.

Graphene-based flexible temperature/pressure dual-mode sensor as a finger sleeve for robotic arms

The design of sensors with multiple sensing functions has become a new challenge to meet the measurement of multiple stimulus signals in people's productive lives. In this article, a sandwich type dual-mode flexible sensor is designed to achieve simultaneous detection of pressure and temperature by constructing different sensing model structures. The dual-mode flexible sensor essentially comprises graphene, melamine sponge and Ag nanoparticle ink. Graphene is coated on the melamine sponge to form a 3D conductive sponge using the dip coating method, while Ag nanoparticle ink is printed on PET to form a serpentine electrode through inkjet printing technology. The sensor has high sensitivity (1.937 × 10−4 Pa−1, R2 = 0.953), a wide operating range (from 0 to 40 kPa), speedy response time (248 ms) and remarkable stability in pressure measurement. It also shows high linearity (TCR = 1.035 × 10−3 °C−1, R2 = 0.987) from 10 to 60 °C and is pressure insensitive in temperature measurement. Experimental results show that the different sensing mechanisms and structural configurations allow the sensors to detect pressure and temperature signals independently or simultaneously without signal coupling. The fabricated sensor has applications in health monitoring, rehabilitation therapy, human-machine interaction and intelligent prosthetics.

Low voltage-driven, high-performance TiO2 thin film transistors with MHz switching speed.

High-speed circuits based on thin film transistors (TFTs) show promising potential applications in biomedical imaging and human-machine interactions. One of the critical requirements for high-speed electronic devices lies in high-frequency switching or amplification at low voltages, typically driven by batteries (∼3.0 V). To date, however, most electrical performances of metal oxide TFTs are measured under direct current (DC) conditions, and their dynamic switching behaviour is scarcely explored and studied systematically. Here in this work, we present low voltage-driven, high-performance TiO2 thin film transistors, which can be operated at a switching speed of MHz. Our proposed TiO2 TFTs demonstrated a high on-off ratio of 107, together with a subthreshold swing (SS) of ∼150 mV Dec-1 averaged over four orders of magnitude, which can be further reduced below 100 mV Dec-1 when the temperature cools to 77 K. Additionally, the TiO2 TFTs exhibit excellent gate-pulse switching at various frequencies ranging from 1.0 Hz to 1.0 MHz. We also explored the potential application of the TiO2 TFTs as logic gates by constructing a resistive-loaded inverter, which shows stable operation at 10 kHz frequency and various temperatures. Thus, our results show the great potential of TiO2 TFTs as a new platform for high-speed electronic applications.

Flexible pressure sensors based on reduced graphene oxide @ porous silicone elastomers with good resolution and wide working range

Current flexible piezoresistive pressure sensors still face challenges, such as poor stability and a limited detection range, making it difficult to accurately discern small-scale pressures in the Pa or MPa range when meeting sudden pressure events. To tackle the above issues, this study presents the development of a competent sensor composed of porous conductive elastomer and laser-induced graphene electrodes. The substrate of the composite elastomer is a porous silicone elastomer obtained by the additive reaction using template methods, and the conductive filler consisting of reduced graphene oxide is synthesized through an eco-friendly approach. We demonstrated that the sensors fabricated here have the ability to precisely resolve pressures from 542 Pa to 1.5 MPa with long-term stability, as well as the detection of subtle signals, such as pulse pulsation and throat vibrations. Above all, such sensors have great potential in the application of human-machine interaction.

A Flexible Self-Powered Noncontact Sensor with an Ultrawide Sensing Range for Human-Machine Interactions in Harsh Environments.

Noncontact human-machine interactions (HMIs) provide a hygienic and intelligent approach to communicate between humans and machines. However, current noncontact HMIs are generally hampered by the interaction distance, and they lack the adaptability to environmental interference such as high humidity conditions. Here, we explore a self-powered electret-based noncontact sensor (ENS) with moisture-resisting ability and ultrawide sensing range exceeding 2.5 m. A megascopic air-bubble structure is designed to enhance charge-storage stability and charge-recovery ability of the ENS based on the heterocharge-synergy effect in electrets. Besides, multilayer electret films are introduced to strengthen the electric field by utilizing the electrostatic field superposition effect. Thanks to the above improved performances of the ENS, we demonstrate various noncontact HMI applications in harsh environments, including noncontact appliances, a moving trajectory and accidental fall tracking system, and a real-time machine learning-assisted gesture recognition system with accuracy as high as 99.21%. This research expands the way for noncontact sensor design and may further broaden applications in noncontact HMIs.

Superelastic Cellulose Sub-Micron Fibers/Carbon Black Aerogel for Highly Sensitive Pressure Sensing.

Superelastic aerogels with rapid response and recovery times, as well as exceptional shape recovery performance even from large deformation, are in high demand for wearable sensor applications. In this study, a novel conductive and superelastic cellulose-based aerogel is successfully developed. The aerogel incorporates networks of cellulose sub-micron fibers and carbon black (SMF/CB) nanoparticles, achieved through a combination of dual ice templating assembly and electrostatic assembly methods. The incorporation of assembled cellulose sub-micron fibers imparts remarkable superelasticity to the aerogel, enabling it to retain 94.6% of its original height even after undergoing 10000 compression/recovery cycles. Furthermore, the electrostatically assembled CB nanoparticles contribute to exceptional electrical conductivity in the cellulose-based aerogel. This combination of electrical conductivity and superelasticity results in an impressive response time of 7.7ms and a recovery time of 12.8ms for the SMF/CB aerogel, surpassing many of the aerogel sensors reported in previous studies. As a proof of concept, the SMF/CB aerogel is utilized to construct a pressure sensor and a sensing array, which exhibit exceptional responsiveness to both minor and substantial human motions, indicating its significant potential for applications in human health monitoring and human-machine interaction.

A stretchable all-nanofiber iontronic pressure sensor

Flexible pressure sensors with high stretchability, sensitivity, and stability are undoubtedly urgently required for potential applications in intelligent soft robots, human-machine interaction, health monitoring, and other fields. However, most current flexible pressure sensors are unable to endure large deformation and are prone to performance degradation or even failure during frequent operation due to their multilayered structures. Here, we propose a stretchable all-nanofiber iontronic pressure sensor that is composed of ionic nanofiber membranes used as dielectric layers and liquid metal used as electrodes. This sensor exhibits a high sensitivity of 1.08 kPa-1 over a wide range of 0-300 kPa, with a fast response-relaxation time of about 18/22 ms and excellent stability. The high sensitivity comes from the electric double layer formed at the ionic film/electrode interface, while high stretchability and stability are enabled by in-situ encapsulated all-nanofiber structures. As a proof of concept, a prototype sensor array is integrated into a soft pneumatic gripper, demonstrating its capability of pressure perception and object recognition during the grasping process. Thus, the scheme provides another excellent strategy to fabricate stretchable pressure sensors with superb performance in terms of high stretchability, sensitivity, and stability.

A soft tactile sensing array with high sensitivity based on MWCNTs-PDMS composite of hierarchically porous structure

The advancement of wearable tactile sensors that involves with high sensitivity under ultra-low pressures is crucial for varieties of human-machine interactive applications, like smart phones, healthcare monitoring, and electronic skins. Here in this paper, a soft capacitive tactile sensing array is introduced based on hierarchically porous multi-walled carbon nanotubes (MWCNTs)-polydimethylsiloxane composite, which leads to sensitivity improvement attributing to a synergistic effect of the hierarchically porous elastomer and conductive MWCNTs supplements. The proposed device exhibits superior pressure-sensing performances, with high sensitivity (3.58 kPa−1) under small mechanical stimuli (<80 Pa), broad measuring range (0–265 kPa), fast response time (<45 ms), good repeatability, minimum limit of detection (<10 Pa), as well as low-hysteresis, allowing efficient sensing of pressure from all types of sources, from vulnerable signals such as human breathing, artery and venous pulses, and soft human finger touch to possible brutal variations such as sudden change of object weight or prompt collide. Moreover, extensive body attached experiments confirm that the soft tactile sensing array is fully human compatible and capable for a variety of human-machine interfaces and health monitoring applications.

Textile-integrated MoS2-PDMS single electrode triboelectric nanogenerator for vibrational energy harvesting and biomechanical motion sensing

With the increasing demand for portable electronics, durable and flexible textile-based single electrode triboelectric nanogenerators (TENGs) have received a significant amount of attention. In this work, we have proposed a flexible textile based single electrode TENG (STENG) composed of MoS2-PDMS composite film. The effect of the concentration (0–10 wt%) of MoS2 in PDMS on the output performance of the as-fabricated devices was systematically studied. The STENG with a 6 wt% concentration of MoS2 exhibits the best output performance with an open-circuit output voltage, short-circuit output current density, and maximum instantaneous output power density of ∼ 320 V, ∼ 15.4 µA/cm2, and ∼ 3.2 mW/cm2, respectively. The enhancement in the output characteristics of the STENG can be explained by the formation of network of nanocapacitors, where the layers of MoS2 behave as electrodes and the insulating PDMS behaves as a nanodielectric. The powering capability of the as-fabricated STENG was tested by charging commercial capacitors and directly flashing 100 commercially available LEDs without any external charge storage component. The textile-based MoS2-PDMS composite is not only used as STENG, but also as a human motion sensor capable of detecting diverse human actions, including wrist bending, arm bending, and leg bending. As a result of its flexible structure, scalable manufacturing process, and ability to sense human motions, the developed MoS2-PDMS composite STENG has promising applications in self-powered wearable electronics and human-machine interaction.

Laser-direct-writing assisted preparation of flexible humidity sensors based on semi-interpenetrating polymer network for applications in non-contact human-machine interaction

Flexible sensors for non-contact human-machine interaction (HMI) received great attention recently due to increasing requirements for cross-infection of bacteria or virus, which is prominent considering the disaster brought about by COVID-19. Herein, a semi-interpenetrating network of poly(dimethyl diallyl ammonium chloride)/crosslinked poly(vinyl alcohol)/graphene oxide (PDDA/c-PVA/GO) was deposited on polytetrafluoroethylene membrane with interdigital electrodes constructed by reducing GO with laser-direct-writing technology to prepare a flexible humidity sensor (FHS). As-prepared PDDA/c-PVA/GO FHS showed excellent sensing performance, including high response magnitude over wide range of relative humidity (RH) (impedance change of three orders of magnitude between 10 % and 90 % RH), good sensing linearity (R2 = 0.991) on a semi-logarithmic scale, small hysteresis (∼ 4.4 % RH) and resistance to humid environment. Moreover, the FHS exhibited good performance in various scenarios of non-contact HMI, including prompt differentiation of respiration status, speech recognition, accurate measurement of fingertip movement (as small as 2 mm). Specially, a 3 × 3 FHS sensor array could realize accurate non-contact positioning. This work provides a new approach for developing high performance flexible sensors with great application potentials in non-contact HMI.

Ti3C2Tx MXene-Based Multifunctional Tactile Sensors for Precisely Detecting and Distinguishing Temperature and Pressure Stimuli.

Although skin-like sensors that can simultaneously detect various physical stimuli are of fair importance in cutting-edge human-machine interaction, robotic, and healthcare applications, they still face challenges in facile, scalable, and cost-effective production using conventional active materials. The emerging two-dimensional transition metal carbide, Ti3C2Tx MXene, integrated with favorable thermoelectric properties, metallic-like conductivity, and a hydrophilic surface, is promising for solving these problems. Herein, skin-like multifunctional sensors are designed to precisely detect and distinguish temperature and pressure stimuli without cross-talk by decorating elastic and porous substrates with MXene sheets. Because the combination of the thermoelectric and conductive MXene with the thermally insulating, elastic, and porous substrate integrates efficient Seebeck and piezoresistive effects, the resultant sensor exhibits not only an ultralow detection limit (0.05 K), high signal-to-noise ratio, and excellent cycling stability for temperature detection but also high sensitivity, fast response time, and outstanding durability for pressure detection. Based on the impressive dual-mode sensing properties and independent temperature and pressure detections, a multimode input terminal and an electronic skin are created, exhibiting great potential in robotic and human-machine interaction applications. This work provides a scalable fabrication of multifunctional tactile sensors for precisely detecting and distinguishing temperature and pressure stimuli.

Multifunctional high-density ultrasonic microsensor for ranging and 2D imaging

Ultrasound has been implemented as a powerful tool for noninvasive imaging and non-contact ranging. However, the size and single-function of traditional ultrasonic probes have limited their adaptability to complex application scenarios in the artificial intelligence era. This paper reports a multifunctional microsensor based on a piezoelectric micromachined ultrasonic transducer (pMUT) array that can be used for ranging and 2D imaging at the same time, in air, with a small working size (2 × 0.8 mm2) and low driving voltage (10 V). A back-end circuit was also designed to optimize the performance of the pMUT array, resulting in a peak-to-peak value of 158 mV. The characterization of the multifunctional microsensor demonstrated that it is capable of performing ranging and 2D imaging simultaneously with high spatial resolution (∼1 mm in ranging and ∼0.2 × 0.2 mm2 in 2D-imaging), even in complex application scenarios. This paper describes a system that integrates micro-electro-mechanical systems device, back-end circuit, imaging algorithm, and denoising algorithm, which allow the microsensor to provide multiple functions. This makes it a promising tool for a wide range of human–machine interaction applications, such as gesture recognition, robotics, and healthcare.

Stretchable, Adhesive, and Bioinspired Visual Electronic Skin with Strain/Temperature/Pressure Multimodal Non-Interference Sensing.

It is highly desirable to construct a single-multimodal sensor that could synchronously perceive multiple stimuli without interference. Here, we propose an adhesive multifunctional chromotropic electronic skin (MCES) that can respond to and distinguish three different stimuli of stain, temperature, and pressure within the two-terminal sensing unit. The mutually discriminating "three-in-one" device converts strain into capacitance and pressure into voltage signals for a tactile stimulus response and produces visual color changes against temperature. In this MCES system, the interdigital capacitor sensor shows high linearity (R2 = 0.998), and temperature sensing is realized via reversible multicolor switching bioinspired by the chameleon, showing attractive potential in visualization interaction. Notably, the energy-harvesting triboelectric nanogenerator in MCES can not only detect pressure incentive but also identify objective material species. Looking forward, these findings promise for multimodal sensor technology with reduced complexity and production costs that are highly anticipated in soft robotics, prosthetics, and human-machine interaction applications.