Human Interface Research Articles

Usability Testing of Novel IoT-Infused Digital Services on Farm Equipment Reveals Farmer’s Requirements towards Future Human–Machine Interface Design Guidelines

The application of digital technologies in the agricultural sector is increasing. One of the new key technologies is the Internet of Things (IoT), which can facilitate the everyday work of farmers. For the successful adoption of IoT-enabled digital products and to ensure improved workflows, the usability of human–machine interfaces is highly important. Various design approaches of human–machine interfaces (HMIs) can currently be found in the monitoring of agricultural machinery. In this work, the most well-known HMIs are considered. Based on a usability test (participants n = 9), the user interface (UI) of a novel mobile application (NEVONEX Cockpit App) was chosen as an example of a design approach of an IoT ecosystem that is oriented towards the UI design of mobile applications. This work aims to identify the weak points of this UI. Conclusions about the needs, and thus an improvement of the user experience, are based on the suggestions for improvement and the information about the general requirements of farmers for a UI for agricultural machinery. Here, it was found that most farmers are satisfied with the UI design of their familiar tractor monitors. According to the survey, the three most important points to be considered in the UI design are intuitive operation and menu navigation, easy access to the essential functions and buttons, and sufficiently large control panels. The conducted usability tests can be considered a basis for developing HMIs for agriculture machinery. Through repeated execution of comparable usability tests, there is the possibility of developing future HMI guidelines for agriculture to improve the user experience (UX). For the NEVONEX ecosystem, feedback from the user interface testing was incorporated in a major revision of the Cockpit App’s design, where a lot more display space was given to the agronomic digital services by smartly arranging infrastructure functions in tiles.

Overview of Gesture Control Technologies

The dynamic development and rapid updating of information and communication technologies (ICT) has created a suitable environment for gesture control of modern computer and robot systems. It is noted that gestures are a necessary component in the communication process. At the same time, it is possible to control the equipment without contact with the support of methods that allow to select and recognize hand gestures. Sufficient work has been done and achievements have been made in the direction of automation and gesture control of technology. The study notes that such systems are more flexible and can easily adapt to people's changing needs. Building a human-machine interface speeds up the communication process and expands the user's capabilities. Gesture control technologies for computer and robotic systems have great potential. A comparative analysis of the parameters of gesture-controlled technologies is shown in the table.

Attention Pyramid Convolutional Neural Network Optimized with Big Data for Teaching Aerobics

Aerobics has emerged as a widely embraced cardiovascular exercise, fostering improved fitness through rhythmic movements that enhance heart rate, stamina, endurance, and cardiovascular health. Effective instruction by skilled professionals is crucial for maximizing the benefits of aerobics, ensuring participants' correct and safe performance. This study introduces the concept of Aerobics Movement Teaching, emphasizing its pivotal role in college physical education. The proposed method, Attention Pyramid Convolutional Neural Network optimized with big data for teaching Aerobics (AP-CNN-BTA), focuses on enhancing aesthetic ability and overall human development. Data from the Simple Ocean Data Assimilation Data Set are collected, preprocessed, and utilized in the teaching process using the Attention Pyramid Convolutional Neural Network, specifically designed for efficient aerobics instruction in coastal areas. The resulting data are stored in the cloud and accessed through a human interface unit. The implementation, carried out in Python, undergoes evaluation using various metrics, including accuracy, computational time, sensitivity, specificity, precision, and ROC analysis. The simulation results demonstrate a remarkable improvement, with the proposed technique achieving 36.52%, 39.55%, and 43.75% higher accuracy compared to existing methods such as Exploration on and thinking about aesthetic infiltration in aerobics teaching in colleges and universities(ETAI-AT-CU), sea level height depending on big data of the Internet of Things along aerobics teaching in coastal regions(SLH-BD-IOT-ATC), and impact of deep learning-based curriculum's ideological and political integration on sports aerobics instruction design(IP-CDL-TDTA), respectively. This research contributes to advancing the efficacy of aerobics instruction, particularly in coastal regions, and underscores the significance of comprehensive student development through high-quality education.

A human machine interface (HMI) assisted portable device for measuring soil efflux using low-cost sensors: design, development and field evaluation

A human machine interface (HMI) assisted portable device for measuring soil efflux using low-cost sensors: design, development and field evaluation

Versatile hybrid transparent triboelectric nanogenerator-solar cell device enables efficient energy harvesting and personal handwriting recognition

The continuous energy supply for wearable electronic devices, including health monitoring sensors and human-machine interface devices, poses a significant challenge to their development. Renewable energy sources, such as nanogenerators and solar cells, offer promising solutions to this challenge. In this study, we explore the potential of a versatile device based on a smart hybrid triboelectric nanogenerator (SH TENG) and photovoltaic (PV) cell for efficient energy harvesting and personal handwriting recognition. A transparent, self-cleaning, UV-protective, bidirectional, non-rigid, lightweight, ultra-durable, eco-friendly, and uniform composite of epoxy resin/ZnO/Ag nanowires-based sandwich-like TENG can produce a power density of 2.3 W m−2. Integrating SH TENG with PV led to improved energy harvesting and increased the charging voltage of the capacitor by up to 5 times. Moreover, this material is a multifunctional sensor capable of detecting temperature, humidity, pressure, handwriting content, and the writer. These features make it an intriguing candidate for e-skin applications. By employing a generalized deep neural network (DNN) model, our device can accurately recognize handwritten letters, numbers, and personal identification with a high level of precision, achieving an accuracy of 97 %. This device can function as an intelligent touch screen with dual-purpose energy production and detection, paving the way for developing highly efficient and sustainable wearable electronic devices.

Monitoring Equipment Malfunctions in Composite Material Machining: Acoustic Emission-Based Approach for Abrasive Waterjet Cutting

This paper introduces an Acoustic Emission (AE)-based monitoring method designed for supervising the Abrasive Waterjet Cutting (AWJC) process, with a specific focus on the precision cutting of Carbon Fiber-Reinforced Polymer (CFRP). In industries dealing with complex CFRP components, like the aerospace, automotive, or medical sectors, preventing cutting system malfunctions is very important. This proposed monitoring method addresses issues such as reductions or interruptions in the abrasive flow rate, the clogging of the cutting head with abrasive particles, the wear of cutting system components, and drops in the water pressure. Mathematical regression models were developed to predict the root mean square of the AE signal. The signal characteristics are determined, considering key cutting parameters like the water pressure, abrasive mass flow rate, feed rate, and material thickness. Monitoring is conducted at both the cutting head and on the CFRP workpiece. The efficacy of the proposed monitoring method was validated through experimental tests, confirming its utility in maintaining precision and operational integrity in AWJC processes applied to CFRP materials. Integrating the proposed monitoring technique within the framework of digitalization and Industry 4.0/5.0 establishes the basis for advanced technologies such as Sensor Integration, Data Analytics and AI, Digital Twin Technology, Cloud and Edge Computing, MES and ERP Integration, and Human-Machine Interface. This integration enhances operational efficiency, quality control, and predictive maintenance in the AWJC process.

An Intelligent Human–Machine Interface Architecture for Long-Term Remote Robot Handling in Fusion Reactor Environments

This paper addresses the intricate challenge posed by remote handling (RH) operations in facilities with operational lifespans surpassing 30 years. The extended RH task horizon necessitates a forward-looking strategy to accommodate the continuous evolution of RH equipment. Confronted with diverse and evolving hardware interfaces, a critical requirement emerges for a flexible and adaptive software architecture based on changing situations and past experiences. The paper explores the inherent challenges associated with sustaining and upgrading RH equipment within an extended operational context. In response to this challenge, a groundbreaking, flexible, and maintainable human–machine interface (HMI) architecture named MAMIC is designed, guaranteeing seamless integration with a diverse range of RH equipment developed over the years. Embracing a modular and extensible design, the MAMIC architecture facilitates the effortless incorporation of new equipment without compromising system integrity. Moreover, by adopting this approach, nuclear facilities can proactively steer the evolution of RH equipment, guaranteeing sustained performance and compliance throughout the extended operational lifecycle. The proposed adaptive architecture provides a scalable and future-proof solution, addressing the dynamic landscape of remote handling technology for decades.

Kinematic Modeling and Performance Analysis of a 5-DoF Robot for Welding Applications

Robotic manipulators are critical for industrial automation, boosting productivity, quality, and safety in various production applications. Key factors like the payload, speed, accuracy, and reach define robot performance. Optimizing these factors is crucial for future robot applications across diverse fields. While 6-Degrees-of-Freedom (DoF)-articulated robots are popular due to their diverse applications, this research proposes a novel 5-DoF robot design for industrial automation, featuring a combination of three prismatic and two revolute (2R) joints, and analyzes its workspace. The proposed techno-economically efficient design offers control over the robot manipulator to achieve any reachable position and orientation within its workspace, replacing traditional 6-DoF robots. The kinematic model integrates both parallel and serial manipulator principles, combining a Cartesian mechanism with rotational mechanisms. Simulations demonstrate the end effector’s flexibility for tasks like welding, additive manufacturing, and material inspections, achieving the desired position and orientation. The research encompasses the design of linear and rotational actuators, kinematic modeling, Human–Machine Interface (HMI) development, and welding application integration. The developed robot demonstrates a superior performance and user-friendliness in welding. The experimental work validates the design’s optimized joint trajectories, efficient power usage, singularity avoidance, easy access in application areas, and reduced costs due to fewer actuators.

Designing Actuated Medical Robots for Safe Human Interaction – A Real Challenge

Abstract The paper studies the influence of human factor and clinical constraints specific to mechanical devices for medical purpose. environment (with a limited workspace, inside or outside the patient body). A medical robot is usually a complex system including mechanical structure, human-machine interface (HMI), electronic components, and software controller. All these components are integrated to perform various tasks for daily medical applications. Because any failure may become very critical, medical robots must function safely and with high reliability, so that designing medical robots for safe human interaction can be considered a real challenge. In order to respect the clinical constrains, a safety real-time controller has been designed.

A Multipurpose Human–Machine Interface via 3D-Printed Pressure-Based Force Myography

A Multipurpose Human–Machine Interface via 3D-Printed Pressure-Based Force Myography

Biomimic and bioinspired soft neuromorphic tactile sensory system

The progress in flexible and neuromorphic electronics technologies has facilitated the development of artificial perception systems. By closely emulating biological functions, these systems are at the forefront of revolutionizing intelligent robotics and refining the dynamics of human–machine interactions. Among these, tactile sensory neuromorphic technologies stand out for their ability to replicate the intricate architecture and processing mechanisms of the brain. This replication not only facilitates remarkable computational efficiency but also equips devices with efficient real-time data-processing capability, which is a cornerstone in artificial intelligence evolution and human–machine interface enhancement. Herein, we highlight recent advancements in neuromorphic systems designed to mimic the functionalities of the human tactile sensory system, a critical component of somatosensory functions. After discussing the tactile sensors which biomimic the mechanoreceptors, insights are provided to integrate artificial synapses and neural networks for advanced information recognition emphasizing the efficiency and sophistication of integrated system. It showcases the evolution of tactile recognition biomimicry, extending beyond replicating the physical properties of human skin to biomimicking tactile sensations and efferent/afferent nerve functions. These developments demonstrate significant potential for creating sensitive, adaptive, plastic, and memory-capable devices for human-centric applications. Moreover, this review addresses the impact of skin-related diseases on tactile perception and the research toward developing artificial skin to mimic sensory and motor functions, aiming to restore tactile reception for perceptual challenged individuals. It concludes with an overview of state-of-the-art biomimetic artificial tactile systems based on the manufacturing–structure–property–performance relationships, from devices mimicking mechanoreceptor functions to integrated systems, underscoring the promising future of artificial tactile sensing and neuromorphic device innovation.

Stretchable MXene/Carbon Nanotube Bilayer Strain Sensors with Tunable Sensitivity and Working Ranges.

Stretchable strain sensors have gained increasing popularity as wearable devices to convert mechanical deformation of the human body into electrical signals. Two-dimensional transition metal carbides (Ti3C2Tx MXene) are promising candidates to achieve excellent sensitivity. However, MXene films have been limited in operating strain ranges due to rapid crack propagation during stretching. In this regard, this study reports MXene/carbon nanotube bilayer films with tunable sensitivity and working ranges. The device is fabricated using a scalable process involving spray deposition of well-dispersed nanomaterial inks. The bilayer sensor's high sensitivity is attributed to the cracks that form in the MXene film, while the compliant carbon nanotube layer extends the working range by maintaining conductive pathways. Moreover, the response of the sensor is easily controlled by tuning the MXene loading, achieving a gauge factor of 9039 within 15% strain at 1.92 mg/cm2 and a gauge factor of 1443 within 108% strain at 0.55 mg/cm2. These tailored properties can precisely match the operation requirements during the wearable application, providing accurate monitoring of various body movements and physiological activities. Additionally, a smart glove with multiple integrated strain sensors is demonstrated as a human-machine interface for the real-time recognition of hand gestures based on a machine-learning algorithm. The design strategy presented here provides a convenient avenue to modulate strain sensors for targeted applications.

Mucus production, host-microbiome interactions, hormone sensitivity, and innate immune responses modeled in human cervix chips

Modulation of the cervix by steroid hormones and commensal microbiome play a central role in the health of the female reproductive tract. Here we describe organ-on-a-chip (Organ Chip) models that recreate the human cervical epithelial-stromal interface with a functional epithelial barrier and production of mucus with biochemical and hormone-responsive properties similar to living cervix. When Cervix Chips are populated with optimal healthy versus dysbiotic microbial communities (dominated by Lactobacillus crispatus and Gardnerella vaginalis, respectively), significant differences in tissue innate immune responses, barrier function, cell viability, proteome, and mucus composition are observed that are similar to those seen in vivo. Thus, human Cervix Organ Chips represent physiologically relevant in vitro models to study cervix physiology and host-microbiome interactions, and hence may be used as a preclinical testbed for development of therapeutic interventions to enhance women’s health.

Programmed multimaterial assembly by synergized 3D printing and freeform laser induction

In nature, structural and functional materials often form programmed three-dimensional (3D) assembly to perform daily functions, inspiring researchers to engineer multifunctional 3D structures. Despite much progress, a general method to fabricate and assemble a broad range of materials into functional 3D objects remains limited. Herein, to bridge the gap, we demonstrate a freeform multimaterial assembly process (FMAP) by integrating 3D printing (fused filament fabrication (FFF), direct ink writing (DIW)) with freeform laser induction (FLI). 3D printing performs the 3D structural material assembly, while FLI fabricates the functional materials in predesigned 3D space by synergistic, programmed control. This paper showcases the versatility of FMAP in spatially fabricating various types of functional materials (metals, semiconductors) within 3D structures for applications in crossbar circuits for LED display, a strain sensor for multifunctional springs and haptic manipulators, a UV sensor, a 3D electromagnet as a magnetic encoder, capacitive sensors for human machine interface, and an integrated microfluidic reactor with a built-in Joule heater for nanomaterial synthesis. This success underscores the potential of FMAP to redefine 3D printing and FLI for programmed multimaterial assembly.

Scalable-produced micro-elastic triboelectric sensing ground for all-weather large-scale applications

Large-scale smart floor will be one potential form for a human–machine interface, but is also facing many difficulties and challenges in the materials science and information engineering. Here, we present a straightforward method for fabricating large-scale micro-elastic triboelectric nanogenerator (ME-TENG) arrays, which can be assembled as triboelectric sensing ground for real-time interface interaction and all-weather monitoring. ME-TENG unit of 6 by 6 arrays offers a maximum power density of 5.84 mW·m−2 under a load resistance of 70 MΩ. It displays a sensitivity of 132.48 mV/kPa in the pressure range of 0 to 180 kPa. The electrical output signals have remained stable in the durability tests over 10 days with various weather conditions. Furthermore, successfully demonstrated a regional comprehensive security monitoring (RCSM) system for enabling all-weather monitoring of the features, position, and movement trajectories of an object. Even more gratifying is that the ME-TENG unit of 30 cm by 30 cm prepared by a simple and fast “all-in-one” method, takes costs of 14.84 dollars/m2 and the production period of 2 h, compared with the most cost-effective piezoelectric sensors, the cost and duration were reduced by 98.67 % and 42.86 %, respectively. This work will greatly contribute to the development and implementation of large-scale sensing and the multidimensional security system.

Correction: Platform Supporting Intelligent Human–Machine Interface (HMI) Applications for Smart Machine Tools

Correction: Platform Supporting Intelligent Human–Machine Interface (HMI) Applications for Smart Machine Tools

A Wearable Bidirectional Human–Machine Interface: Merging Motion Capture and Vibrotactile Feedback in a Wireless Bracelet

Humans interact with the environment through a variety of senses. Touch in particular contributes to a sense of presence, enhancing perceptual experiences, and establishing causal relations between events. Many human–machine interfaces only allow for one-way communication, which does not do justice to the complexity of the interaction. To address this, we developed a bidirectional human–machine interface featuring a bracelet equipped with linear resonant actuators, controlled via a Robot Operating System (ROS) program, to simulate haptic feedback. Further, the wireless interface includes a motion sensor and a sensor to quantify the tightness of the bracelet. Our functional experiments, which compared stimulation with three and five intensity levels, respectively, were performed by four healthy participants in their twenties and thirties. The participants achieved an average accuracy of 88% estimating three vibration intensity levels. While the estimation accuracy for five intensity levels was only 67%, the results indicated a good performance in perceiving relative vibration changes with an accuracy of 82%. The proposed haptic feedback bracelet will facilitate research investigating the benefits of bidirectional human–machine interfaces and the perception of vibrotactile feedback in general by closing the gap for a versatile device that can provide high-density user feedback in combination with sensors for intent detection.

A conductive and anti-freezing gelatin-PAA-based organic hydrogel (PC-OH) with high adhesion and self-healing activities for wearable electronics

In the realm of flexible wearable electronics and interactive technologies, the adhesive and touch-sensing properties of materials play a crucial role, especially soft ionic conductors. However, these materials face several challenges in practical application, including structural damage, loss of functionality, and device stratification, particularly in extreme environments. To address this, a novel multifunctional organic hydrogel, termed PC-OH, was designed and prepared to enhance its performance. The hydrogel exhibits self-adhesion, self-healing, anti-freezing, anti-drying, and conductive capabilities. It demonstrates outstanding self-adhesive properties on skin surfaces under air (approximately 967 J/m2) or underwater (approximately 428 J/m2) conditions, with high transparency (visible light transmittance of 87 %), anti-freezing capability (−60 °C), anti-drying capability (60 °C), rapid self-healing in air or water (within 4 s), long-term stability over a wide temperature range (−20 °C to 60 °C for up to 7 days), high extensibility (up to ∼1700 % at 60 °C and ∼1200 % at −20 °C), and conductivity over a broad temperature range (ionic conductivity of 1.93 S m−1 at 60 °C and 3.13 × 10-3 S m−1 at −30 °C). Based on these properties of PC-OH hydrogel, we constructed a capacitive touch system to demonstrate its practical application. The touch panel features high stretchability, transparency, anti-freezing, anti-dehydration, self-healing, and self-adhesive human–machine interface capabilities, enabling integration with the skin under abnormal temperature and humidity conditions. Tasks such as writing text, drawing graphics, and playing electronic games validated the significant potential of the hydrogel in touch panels, providing a solution to long-standing issues of adhesion and tactile sensing.

Oculomotor behavior can be adjusted on the basis of artificial feedback signals indicating externally caused errors.

Whether a saccade is accurate and has reached the target cannot be evaluated during its execution, but relies on post-saccadic feedback. If the eye has missed the target object, a secondary corrective saccade has to be made to align the fovea with the target. If a systematic post-saccadic error occurs, adaptive changes to the oculomotor behavior are made, such as shortening or lengthening the saccade amplitude. Systematic post-saccadic errors are typically attributed internally to erroneous motor commands. The corresponding adaptive changes to the motor command reduce the error and the need for secondary corrective saccades, and, in doing so, restore accuracy and efficiency. However, adaptive changes to the oculomotor behavior also occur if a change in saccade amplitude is beneficial for task performance, or if it is rewarded. Oculomotor learning thus is more complex than reducing a post-saccadic position error. In the current study, we used a novel oculomotor learning paradigm and investigated whether human participants are able to adapt their oculomotor behavior to improve task performance even when they attribute the error externally. The task was to indicate the intended target object among several objects to a simulated human-machine interface by making eye movements. The participants were informed that the system itself could make errors. The decoding process depended on a distorted landing point of the saccade, resulting in decoding errors. Two different types of visual feedback were added to the post-saccadic scene and we compared how participants used the different feedback types to adjust their oculomotor behavior to avoid errors. We found that task performance improved over time, regardless of the type of feedback. Thus, error feedback from the simulated human-machine interface was used for post-saccadic error evaluation. This indicates that 1) artificial visual feedback signals and 2) externally caused errors might drive adaptive changes to oculomotor behavior.

Piezotronic Transistors Based on GaN Wafer for Highly Sensitive Pressure Sensing with High Linearity and High Stability.

Piezotronic effect utilizing strain-induced piezoelectric polarization to achieve interfacial engineering in semiconductor nanodevices exhibits great advantages in applications such as human-machine interfacing, micro/nanoelectromechanical systems, and next-generation sensors and transducers. However, it is a big challenge but highly desired to develop a highly sensitive piezotronic device based on piezoelectric semiconductor wafers and thus to push piezotronics toward wafer-scale applications. Here, we develop a bicrystal barrier-based piezotronic transistor for highly sensitive pressure sensing by p-GaN single-crystal wafers. Its pressure sensitivity can be as high as 19.83 meV/MPa, which is more than 15 times higher than previous bulk-material-based piezotronic transistors and reaches the level of nanomaterial-based piezotronic transistors. Moreover, it can respond to a very small strain of 3.3 × 10-6 to 1.1 × 10-5 with high gauge factors of 1.45 × 105 to 1.38 × 106, which is a very high value among various strain sensors. Additionally, it also exhibits high stability (current stability of 97.32 ± 2.05% and barrier height change stability of 95.85 ± 3.43%) and high linearity (R2 ∼ 0.997 ± 0.002) in pressure sensing. This work proves the possibility of designing a bicrystal barrier as the interface to obtain a strong piezotronic effect and highly sensitive piezotronic devices based on wafers, which contributes to their applications.