Human-machine Interface Technology Research Articles

Ultra-Fast Fabrication of Mechanical-Water-Responsive Color-Changing Photonic Crystals Elastomers and 3D Complex Devices.

The traditional fabrication of opal-structured photonic crystals is constrained by the rate of solvent evaporation, a process that is not only time-consuming but also labor-intensive. This study introduces a paradigm shift by incorporating silica nanoparticles (SiNPs) with high zeta potentials and hydrogen bonding capabilities into an elastomeric matrix, resulting in a novel non-close-packed structure. This innovation circumvents the limitations of conventional methods by enabling the rapid formation of photonic inks (PI) into vibrant and luminous photonic elastomers (PEs) within seconds. These PEs demonstrate remarkable mechanochromic properties, exhibiting dynamic color changes across the visible spectrum in response to tensile and compressive deformations. Furthermore, the presence of hydroxyl groups endows the PEs with superior water-responsiveness, which can be finely tuned through the ink formulation. The elimination of solvent evaporation dependency facilitates the fabrication of macroscopic photonic crystal devices with complex geometries using digital light processing (DLP)-based 3D printing. This approach ensures exceptional optical performance and high customization potential. The resulting PEs hold significant promise for applications in smart wearables, soft robotics, and advanced human-machine interface technologies.

Discovering the Power of External Human-Machine Interface: An EEG Study on the Driving Anger Regulation

Driving with anger can result in hazardous driving behaviors, often arising from inadequate communication among Traffic Participants (TPs). Currently, external Human-Machine Interface (eHMI) technology provides a novel channel for enhancing communication among TPs, yet its role in regulating driving emotions remains uncertain. This study investigates the impact of eHMI technology on the regulation of driving anger. We simulated a driving scenario in which the preceding vehicle abruptly applied brakes to elicit anger from the following driver. Three eHMI formats (Symbol, Text, and Symbol + Text) positioned at the rear of the preceding vehicle were implemented to explain the sudden braking event and regulate drivers’ emotions. Objective electroencephalogram (EEG) signals and subjective assessment data were analyzed to assess the effects of eHMIs on driving anger regulation. The results indicate that driving with eHMIs exhibited significant regulatory quality of drivers’ anger. Among the three formats, Symbol + Text eHMI exerted a notable regulatory impact, while both Symbol and Text eHMI contributed to emotional regulation. This study illustrates the positive impact of eHMI on emotion regulation in traffic environments, providing valuable backing for traffic management authorities in developing guidelines and policies, and contributing to the standardization of future eHMI technologies.

Hyperparameter tuning using Lévy flight and interactive crossover-based reptile search algorithm for eye movement event classification.

Introduction: Eye movement is one of the cues used in human-machine interface technologies for predicting the intention of users. The developing application in eye movement event detection is the creation of assistive technologies for paralyzed patients. However, developing an effective classifier is one of the main issues in eye movement event detection. Methods: In this paper, bidirectional long short-term memory (BILSTM) is proposed along with hyperparameter tuning for achieving effective eye movement event classification. The Lévy flight and interactive crossover-based reptile search algorithm (LICRSA) is used for optimizing the hyperparameters of BILSTM. The issues related to overfitting are avoided by using fuzzy data augmentation (FDA), and a deep neural network, namely, VGG-19, is used for extracting features from eye movements. Therefore, the optimization of hyperparameters using LICRSA enhances the classification of eye movement events using BILSTM. Results and Discussion: The proposed BILSTM-LICRSA is evaluated by using accuracy, precision, sensitivity, F1-score, area under the receiver operating characteristic (AUROC) curve measure, and area under the precision-recall curve (AUPRC) measure for four datasets, namely, Lund2013, collected dataset, GazeBaseR, and UTMultiView. The gazeNet, human manual classification (HMC), and multi-source information-embedded approach (MSIEA) are used for comparison with the BILSTM-LICRSA. The F1-score of BILSTM-LICRSA for the GazeBaseR dataset is 98.99%, which is higher than that of the MSIEA.

Ensemble Learning Method for the Continuous Decoding of Hand Joint Angles.

Human-machine interface technology is fundamentally constrained by the dexterity of motion decoding. Simultaneous and proportional control can greatly improve the flexibility and dexterity of smart prostheses. In this research, a new model using ensemble learning to solve the angle decoding problem is proposed. Ultimately, seven models for angle decoding from surface electromyography (sEMG) signals are designed. The kinematics of five angles of the metacarpophalangeal (MCP) joints are estimated using the sEMG recorded during functional tasks. The estimation performance was evaluated through the Pearson correlation coefficient (CC). In this research, the comprehensive model, which combines CatBoost and LightGBM, is the best model for this task, whose average CC value and RMSE are 0.897 and 7.09. The mean of the CC and the mean of the RMSE for all the test scenarios of the subjects' dataset outperform the results of the Gaussian process model, with significant differences. Moreover, the research proposed a whole pipeline that uses ensemble learning to build a high-performance angle decoding system for the hand motion recognition task. Researchers or engineers in this field can quickly find the most suitable ensemble learning model for angle decoding through this process, with fewer parameters and fewer training data requirements than traditional deep learning models. In conclusion, the proposed ensemble learning approach has the potential for simultaneous and proportional control (SPC) of future hand prostheses.

Theoretical Basis of Biomimetic Flexible Piezoelectric Acoustic Sensors for Future Customized Auditory Systems

AbstractFlexible piezoelectric sensors have been spotlighted as an essential human–machine interface (HMI) by obtaining high‐quality data from omnipresent biomechanical inputs. Because human voice is the most intuitive bio‐signal among them, flexible piezoelectric acoustic sensors (f‐PAS) have a potential to shift the paradigm of HMI technologies. Despite the reported outstanding performance such as high sensitivity and speaker recognition accuracy, the theoretical investigation of f‐PAS has been insufficient to realize future customized development, because sensing principles are fundamentally different from commercialized microphones. Here, a theoretical framework of self‐powered f‐PAS by using mechanical and electrical physics is introduced. First of all, the basic theory of f‐PAS is compared with the auditory system of human cochlear. Based on the biomimetic trapezoidal shape, the resonant frequencies are analyzed with various structural and material conditions. In addition, the piezoelectricity of f‐PAS is derived to predict the sensitivity and SNR prior to experiments. To investigate sensor properties under the medium condition that is similar to human ear, the acoustic responses depending on the states of matter are theoretically compared. Finally, the distance limit of f‐PAS is studied with the correlations between piezoelectricity and sound pressure, which would provide novel strategies of functional material design for future applications of f‐PAS.

A novel equestrian helmet testing method: helmet liner performance in highly realistic simulation

ABSTRACT Objective Employ a novel testing method to assess Multi Directional Impact Protection System (MIPS) helmet technology on rotational velocity and acceleration during head impact. Methods An optimization study was completed utilizing a 50th percentile male Hybrid III anthropomorphic test device (ATD). Helmets included expanded polystyrene foam (EPS) and two different MIPS helmets (MIPS 1, MIPS 2). A 24.38-m-long elevated track with rails and a motorized sled was utilized to replicate a fall from approximately 2.13 m. The sled was set to a speed of 20.92 kph, where a tripping mechanism induced rotation in the ATD from the sled and onto a sand surface. During impact of the ATD with the sand surface, head kinematics were measured using resultant acceleration (peak G’s), duration of impact (ms), and rotational velocity (rad/s). Results A total of three trials for each helmet did not demonstrate a significant difference between the EPS vs. MIPS 1 group with, peak (G’s) for resultant acceleration (p = 0.100), duration (ms) for resultant acceleration, (p = 0.100), peak (G’s) for rotational velocity, (p = 0.700), and duration (ms) for rotational velocity (p = 0.700). Similarly, the EPS vs. MIPS 2 testing demonstrated no significant differences between the MIPS 2 helmet compared to the EPS helmet, with resultant acceleration (p = 0.400), duration acceleration (p = 0.200), rotational velocity (p = 0.400) and duration velocity (p = 0.400). However, when the MIPS helmet data were pooled, and the EPS helmet data were compared, a statistically significant difference in the duration of acceleration was found (p = 0.048). Conclusions Current testing uses a helmeted head form which is dropped or rolled from a prescribed height. These methods discount the loading placed on the neck and head through the angular momentum of the body. Our novel testing method did not find significant differences between the helmet types in diminishing peak rotational forces to the brain; however, our data suggests that MIPS helmet liners may reduce duration of impact. The reduction of acceleration duration could indicate less rotation of the neck, due to the dampening of these forces by the MIPS liners.

Transforming Education and Learning through Chat GPT: A Systematic Literature Review

Technology is generally interpreted as anything that can provide us with convenience in many ways. Technology also has a huge impact on education and learning. These two things become increasingly inseparable because their roles are interconnected. One of them is by using Chat GPT. The purpose of this research is to explain Transforming Education and Learning through Chat GPT. A review is conducted on the state-of-the-art methods using the preferred reporting items for reviews and meta-analyses. We must know and practice changes in education and learning patterns at all levels of education by describing the characteristics of learning that are currently needed. by using GPT Chat digital technology which has benefits and reasons for using this technology, such as using GPT chat in the transformation of education and learning. GPT chat has many benefits in education and learning but GPT chat is only a tool and the results depend on the data available during training and the model's understanding of the question and context. Although Chat GPT has its benefits, be wise in using the information provided and don't hesitate to verify important information from trusted sources. GPT chat is important because it provides benefits in various fields, such as customer service, education, research, product development, language skills development, and advancing human-machine interface technology

Soft Human-Machine Interface Sensing Displays: Materials and Devices.

The development of human-interactive sensing displays (HISDs) that simultaneously detect and visualize stimuli is important for numerous cutting-edge human-machine interface technologies. Therefore, innovative device platforms with optimized architectures of HISDs combined with novel high-performance sensing and display materials are demonstrated. This study comprehensively reviews the recent advances in HISDs, particularly the device architectures that enable scaling-down and simplifying the HISD, as well as material designs capable of directly visualizing input information received by various sensors. Various HISD platforms for integrating sensors and displays are described. HISDs consist of a sensor and display connected through a microprocessor, and attempts to assemble the two devices by eliminating the microprocessor are detailed. Single-device HISD technologies are highlighted in which input stimuli acquired by sensory components are directly visualized with various optical components, such as electroluminescence, mechanoluminescence and structural color. The review forecasts future HISD technologies that demand the development of materials with molecular-level synthetic precision that enables simultaneous sensing and visualization. Furthermore, emerging HISDs combined with artificial intelligence technologies and those enabling simultaneous detection and visualization of extrasensory information are discussed.

A Methodology for the Systematic Selection of Human-Machine Interface Device Types in Production Machinery Development

Progressive digitization accelerates the evolution of human-machine interface (HMI) technologies. In mechanical engineering, HMIs are decisive for accomplishing user-friendly, efficient, and effective machine control. Although various technologies, such as AR-glasses, have already been applied in industrial practice, mechanical engineering companies still face tremendous challenges in selecting and implementing suitable HMI devices. Therefore, this paper presents a novel methodology for systematically evaluating and selecting a suitable HMI device type for production machinery in mechanical engineering product development. The methodology provides a use case-specific process considering the company's resources, innovation strategy, and other relevant criteria enabling the systematic selection of a suitable HMI device type out of a predefined set of existing options. A case study demonstrates the applicability and practicability of the methodology.

Multimodal electronic textiles for intelligent human-machine interfaces

Smart wearable electronic devices capable of information exchanging (such as human-machine interfaces) have developed into key carriers for the interconnection, intercommunication, and interaction between humans and machines. Multimodal electronic textiles that incorporate multifunctional sensors into daily clothing are an emerging technology to realize smart wearable electronics. This has greatly advanced human-machine interface technology by bridging the gap between wearing comfort and traditional wearable electronic devices, which will facilitate the rapid development and wide application of natural human-machine interfaces. In this article, we provide a comprehensive summary of the latest research progress on multimodal electronic textiles for intelligent human-machine interfaces. Firstly, we introduce the most representative electronic textile manufacturing strategies in terms of functional fiber preparation and multimodal textile forming. Then, we explore the multifunctional sensing capability of multimodal electronic textiles and emphasize their advanced applications in intelligent human-machine interfaces. Finally, we present new insights on the future research directions and the challenges faced in practical applications of multimodal electronic textiles.

A Gesture Recognition Strategy Based on A-Mode Ultrasound for Identifying Known and Unknown Gestures

Human-machine interface(HMI) technology has gradually become a research hotspot with the continuous development of computer technology and Internet of Things (IOT) technology. Hand gesture recognition (HGR) as an important part of HMI technology has been widely concerned. Among the many technical routes of HGR technology, the wearable HGR technology based on A-mode ultrasound shows great application potential due to its advantages such as lightweight device, free from sensors’ constraint etc. However, in the process of processing A-mode ultrasonic signal, the amplitude of the signal at the main position may vary due to muscle fatigue and strength etc. when the same gesture is repeated some time later. In this paper, we design a method to overcome this problem and accomplish HGR in offline state by setting the threshold and use normalization method for the energy feature of ultrasonic signal according to the threshold. Three machine learning algorithms, including support vector machine (SVM), linear discriminant analysis (LDA) and Naive Bayes (NB) were used to verify the feasibility of this method, and the average recognition accuracy in the experiment reached 95.26%. Signal stability is also verified to be improved. In addition, We design a recognition strategy based on NB algorithm in this paper so that the model can identify the unknown gestures (i.e. gestures that have not been trained by the model). Five known gestures and five unknown gestures were set in the experiment, the average recognition accuracy of the known gestures is 91.2%, and the unknown gestures is 81.8%. This method can be used to optimize the user experience of HGR system.

Speech Enhancement Framework with Noise Suppression Using Block Principal Component Analysis

With the advancement in voice-communication-based human–machine interface technology in smart home devices, the ability to decompose the received speech signal into a signal of interest and an interference component has emerged as a key requirement for their successful operation. These devices perform their tasks in real time based on the received commands, and their effectiveness is limited when there is a lot of ambient noise in the area in which they operate. Most real-time speech enhancement algorithms do not perform adequately well in the presence of high amounts of noise (i.e., low input-signal-to-noise ratio). In this manuscript, we propose a speech enhancement framework to help these algorithms in situations when the noise level in the received signal is high. The proposed framework performs noise suppression in the frequency domain by generating an approximation of the noisy signals’ short-time Fourier transform, which is then used by the speech enhancement algorithms to recover the underlying clean signal. This approximation is performed by using the proposed block principal component analysis (Block-PCA) algorithm. To illustrate efficacy of the proposed framework, we present a detailed performance evaluation under different noise levels and noise types, highlighting the effectiveness of the proposed framework. Moreover, the proposed method can be used in conjunction with any speech enhancement algorithm to improve its performance under moderate to high noise scenarios.

Fabrication and Performance Evolution of AgNP Interdigitated Electrode Touch Sensor for Automotive Infotainment.

In the present scenario, a considerable assiduity is provided to develop novel human-machine interface technologies that rapidly outpace the capabilities of display technology in automotive industries. It is necessary to use a new cockpit design in conjunction with a fully automated driving environment in order to enhance the driving experience. It can create a seamless and futuristic dashboard for automotive infotainment application. In the present study, an endeavor was made to equip the In-vehicle bezels with printed capacitive sensors for providing superior sensing capabilities. Silver Nanoparticles based interdigitated pattern electrodes were formed over polycarbonate substrates to make printed capacitive sensors using screen printing process. The developed sensor was investigated to evaluate the qualitative and quantitative measures using direct and in-direct contact of touch. The proposed approach for sensors pattern and fabrication can highly impact on sensor performance in automotive infotainment application due to the excellent spatial interpolation with lower cost, light weight, and mechanical flexibility.

Self-powered finger motion-sensing structural color display enabled by block copolymer photonic crystal

Self-powered user-interactive displays that facilitate the visualization of human information acquired by sensors are of great interest in emerging human–machine interface technology with efficient energy consumption. Herein, a self-powered motion-sensing display capable of simultaneously detecting and visualizing finger motions is presented. Our device is based on a one-dimensional photonic crystal of an interpenetrated hydrogel network block copolymer (IHN-BCP) consisting of alternating water-absorbable and non-absorbable lamellae. Triboelectrification is achieved as a function of relative humidity from 30% to 80%. The direct visualization of the humidity is also achieved through the humidity-dependent structural color of the photonic crystal in the full visible range. Furthermore, the humidity-responsive triboelectrification and structural color of our IHN-BCP photonic crystal facilitates the development of a self-powered finger motion-sensing display where diverse gestures of a finger with natural humidity are quantitatively recognized, such as vertical and sliding motion of the finger with simultaneous visualization of the motions in both contact and non-contact modes.

Kontrol Robot Menggunakan Gerakan Mata Berbasis Sinyal Electrooculography (EOG)

Biomedical technology has now been widely adopted as a means of monitoring the human body in real-time. For example, to detect eye movement. In the medical world, eye movement can be used to determine the type of disease. With the application of human-machine interface (HMI) technology, eyeball movement can be developed in the robotics industry as robot navigation. For example, by moving the eyeball left and right, the robot can interpret the eye signal to move left and right. The interaction between the eyeball movement and the robot is of particular concern in this study. This study aimed to design a measuring instrument for eye movement detection using Electrooculography (EOG) techniques to move a wheeled robot. The EOG measuring instrument consisting of an instrument differential amplifier, a low pass filter, and a high pass filter has been applied in this research. The signal generator technique on EOG is carried out by placing electrodes on three sides of the face, namely forehead (G), left horizontal (H-), right horizontal (H +). The experimental results showed a significant difference between the left and right eye movement amplitude signals. This amplitude is used to classify the movement of the robot wheel towards the left and right. The process of sending robot signals and EOG measuring instruments uses Bluetooth HC-05 serial communication. Based on the research results, it is proven that the robot manages to move left and right according to the eyeball movement.

Self-Healable, High-Strength Hydrogel Electrode for Flexible Sensors and Supercapacitors.

Flexible energy storage materials and sensors have become the key equipment of human-machine interface technology. For the preparation of these devices, hydrogel electrodes are relevant because of their unique porous structure, high capacitance, flexibility, small size, and lightweight. In this paper, regular polypyrrole (PPy) is synthesized on a heat-induced phase-separated gel (H-Gel/AS) by the template degradation method, and a gelatin-based PPy hydrogel with high strength, high strain rate, and high conductivity is prepared. Moreover, by adding multiwalled carbon nanotubes (MWCNTs) into a gelatin solution according to the H-Gel/AS method, the electrochemical performance of the resulting H-Gel/AS-MWCNTs-PPy electrode is greatly improved. When the H-Gel/AS-MWCNTs-PPy gel is immersed in an ammonium sulfate solution, wrinkles appear on the surface, resulting in further enhancement of the capacitance. On this basis, a flexible sensor and a solid-state supercapacitor are assembled, and their performance is tested. The sensor can detect tensile, bending, and twisting strains with high sensitivity. Meanwhile, as a flexible solid-state supercapacitor, the specific capacitance is 75 F g-1, and the capacitance retention rate after 5000 cycles is 98.1% under bending conditions. More importantly, the gelatin-based hydrogel shows great potential for application in wearable devices.

Towards Improving the Quality of Electrophysiological Signal Recordings by Using Microneedle Electrode Arrays.

Electrophysiological signals are recorded generally by metal electrodes placed on body surface. For long term use, the signal quality may decay with the change of interface impedance between electrodes and skin due to the conductive hydrogel dehydration. Besides, electrodes may shift during body movements, which causes unstable signal recordings. To improve the quality of electrophysiological signal recordings on human body surface, a type of microneedle electrode array (MEA) with microneedles around 550 μm in length was fabricated with a magnetization-induce self-assembly method. Compared with the commonly used dry electrode array, the MEA has lower and more stable interface impedance, especially when the electrode-skin interface is under unstable pressures. For electrophysiological signal recording, the MEA can acquire electromyography (EMG) with significantly lower noise energy, higher signal-to-noise ratio, and higher motion-classification accuracy based on the EMG pattern-recognition method. Additionally, high quality electrocardiography (ECG) can be recorded by using the MEA, where more accurate R-peaks are extracted in different scenarios. Besides, there was no report about any discomfort like bleeding or inflammation by all the subjects. This research proved that the microneedles on the MEA can penetrate through the corneum and reach the epidermis of the subjects, which could avoid the influence of corneum and fix the electrode on the body surface for high-quality signal recording especially during body movements. Furthermore, the microneedles would not touch the dermis, enabling a painless signal acquisition. This work is beneficial to the applications of wearable human-machine interface technology.

The Display Week 2021 Symposium Preview

The Display Week 2021 Symposium Preview

Helically Intersected Conductive Network Design for Wearable Electronic Devices: From Theory to Application.

Flexible and stretchable strain sensors are crucial components for wearable electronics that can detect and quantify the stimuli from the environment and thus realize the rapid feedback and control of smart devices. However, reconciliation of the conflict between resourceful design of conductive networks and large-scale production in the industry still faces a huge challenge. Herein, we present a new flow-manipulated strategy to prepare a wearable strain sensor featuring a helically intersected conductive network, which exhibited easy integration, multidimensional sensibility, and robust mechanical properties. From visualization of simulation and verification of experimental results, the helically intersected conductive network formed in an elastomer ring can simultaneously reflect the static and dynamic mechanical responses with a tunable gauge factor (10.41-31.12), wide linear region (0-40o), mechanical robustness (σs = ∼7 MPa, ε = ∼1400%), and rapid response time (∼300 ms). We further constructed a control system based on smart rings and demonstrated its application in controlling industrial robotic arms and remote-controlled cars. Looking ahead, this kind of a smart ring will be more widely used in space and underwater exploration, intelligent robotics, and human-machine interface technologies.

Metamaterials-Enabled Sensing for Human-Machine Interfacing.

Our modern lives have been radically revolutionized by mechanical or electric machines that redefine and recreate the way we work, communicate, entertain, and travel. Whether being perceived or not, human-machine interfacing (HMI) technologies have been extensively employed in our daily lives, and only when the machines can sense the ambient through various signals, they can respond to human commands for finishing desired tasks. Metamaterials have offered a great platform to develop the sensing materials and devices from different disciplines with very high accuracy, thus enabling the great potential for HMI applications. For this regard, significant progresses have been achieved in the recent decade, but haven’t been reviewed systematically yet. In the Review, we introduce the working principle, state-of-the-art sensing metamaterials, and the corresponding enabled HMI applications. For practical HMI applications, four kinds of signals are usually used, i.e., light, heat, sound, and force, and therefore the progresses in these four aspects are discussed in particular. Finally, the future directions for the metamaterials-based HMI applications are outlined and discussed.